-- Logs begin at Wed 2022-07-27 19:10:26 UTC, end at Wed 2022-07-27 21:01:59 UTC. -- Jul 27 19:53:37.693528 dv-cinder-851249-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 19:53:37.702851 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 19:53:37 2022] *** Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: nodename: dv-cinder-851249-1-iscsi Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: machine: x86_64 Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: clock source: unix Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: pcre jit disabled Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: detected number of CPU cores: 4 Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: current working directory: / Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: detected binary path: /usr/bin/uwsgi-core Jul 27 19:53:37.704920 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: your processes number limit is 47680 Jul 27 19:53:37.705617 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: your memory page size is 4096 bytes Jul 27 19:53:37.705617 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: detected max file descriptor number: 2048 Jul 27 19:53:37.705617 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: lock engine: pthread robust mutexes Jul 27 19:53:37.705617 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: thunder lock: enabled Jul 27 19:53:37.705617 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 19:53:37.705617 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 19:53:37.744188 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: Python main interpreter initialized at 0x55ebb5fdd9c0 Jul 27 19:53:37.744188 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: python threads support enabled Jul 27 19:53:37.744188 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: your server socket listen backlog is limited to 100 connections Jul 27 19:53:37.744188 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: your mercy for graceful operations on workers is 90 seconds Jul 27 19:53:37.744535 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 19:53:37.744632 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: *** Operational MODE: preforking *** Jul 27 19:53:37.744907 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: *** uWSGI is running in multiple interpreter mode *** Jul 27 19:53:37.744907 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: spawned uWSGI master process (pid: 80322) Jul 27 19:53:37.745109 dv-cinder-851249-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 19:53:37.752402 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: spawned uWSGI worker 1 (pid: 80335, cores: 1) Jul 27 19:53:37.752402 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: spawned uWSGI worker 2 (pid: 80336, cores: 1) Jul 27 19:53:37.752402 dv-cinder-851249-1-iscsi devstack@keystone.service[80322]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 19:53:43.268192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:43.268192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: from cryptography.utils import int_from_bytes Jul 27 19:53:43.269014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:43.269014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: from cryptography.utils import int_from_bytes Jul 27 19:53:43.269421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:43.269421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: from cryptography.utils import int_from_bytes Jul 27 19:53:43.269537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:43.269537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: from cryptography.utils import int_from_bytes Jul 27 19:53:43.318231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 19:53:43.318387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 19:53:43.318471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 19:53:43.318560 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 19:53:43.318639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] command line args: [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 19:53:43.318715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] command line args: [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 19:53:43.318862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 19:53:43.318948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 19:53:43.319026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ================================================================================ {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 19:53:43.319105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ================================================================================ {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 19:53:43.319294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] admin_token = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.319373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] admin_token = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.319542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] config_dir = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.319627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] config_dir = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.319732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.319819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.319897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] config_source = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.319973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] config_source = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.320079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] debug = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.320159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] debug = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.320277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.320626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.320760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.320864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.320991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] list_limit = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] list_limit = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] log_config_append = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.321918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] log_config_append = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] log_dir = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] log_dir = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] log_file = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] log_file = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.322953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.323976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.324968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.325160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.325752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.325912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.326004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.326193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.326314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.326695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.326785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.326864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] publish_errors = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] publish_errors = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.327949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] standard_threads = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] standard_threads = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.328936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] use_journal = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] use_journal = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] use_json = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] use_json = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] use_stderr = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] use_stderr = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] use_syslog = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] use_syslog = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.329950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.330047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:43.330216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.330315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.330419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.330514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.330657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.330760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.330864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.330968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331371 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.331952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.332928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.333998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.334926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.335937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.336980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.337910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338384 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.338937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339271 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.339981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.340975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341336 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.341931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.342046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.342160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.342274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.342387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.342508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.342631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.342753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.343944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.344995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.345974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.346869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.347975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.348130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.348262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.348441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.348569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.348697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.348825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.348964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.349993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.350122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.350249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.350376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.350512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.350640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.350764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.350887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.351945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352336 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.352959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] auth.external = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.353980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] auth.external = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] auth.password = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.354964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] auth.token = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] auth.password = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] auth.token = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.355913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.356893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] credential.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] credential.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.357863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.358811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.359832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.360892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361047 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.361867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.362013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.362090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.362258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.362335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.362494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.362574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.362744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.363005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.363245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.363323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.363503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.363584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.363746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.363826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.364005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.364085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.364246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.364331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.364885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.364974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.365995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.366955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368336 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.368948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369400 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.369892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.370985 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.371062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.371224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.371310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.371478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.371564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.371721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.371855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.372523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.372783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.373021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.373268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.373505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.373747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.373988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.374234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.374475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.374715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.374970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.375223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.375459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.375696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.375948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.376201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.376413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.376497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.376692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.376783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.376931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.377010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.377168 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.377247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.377409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.377497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.377673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.377753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.378074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.378153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.378435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.378561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.378694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.378947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.379026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.379199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.379460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.379565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.379718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.379798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.379960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.380979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.381066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.381412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.381491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.381655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.381899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.381978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.382165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.382374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.382553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.382814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.382891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.383057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.383299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.383380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.383553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.383774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.383853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.384031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.384261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.384351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.384537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.384772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.384862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.385024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.385230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.385310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.385677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.385754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.385968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.386153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.386240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.386451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.386601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.386680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.386937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.387061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.387192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.387447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.387525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.388743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.389067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.389320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.389573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.390018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.390257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.390513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.390593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.390773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.390998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.391078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.391276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.391464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.391543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.391764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.391951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.392030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.392250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.392475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.392604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.392853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.392934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.393207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.393285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.393469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.393691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.393779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.393960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] role.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.394158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.394237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] role.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.394445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.394593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.394704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.394964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.395042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.395222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.395411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.395490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.395719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.395852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.395975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.396113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.396230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.396512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.396605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.396755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.396858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.397854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.398986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] role.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] role.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.399948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.400954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.401938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402493 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.402966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.403128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.403216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.403412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.403603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.403682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.403845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.403923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.404953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.405910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.406857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.connection = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.407991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.408971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.409939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.410976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.411128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.411206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.411373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.411461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.411611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.411690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.411781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 19:53:43.411860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.412071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.412367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.412651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.connection = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.412898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.413145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.413384 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.413627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.413863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.414100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.414335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.414570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.414813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.415048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.415290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.415526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.415774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.416015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.416261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.416519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.416762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.416996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:43.417165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 19:53:43.441617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.441852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.442083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.442879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.443114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.443341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.451012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.452250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.480423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.480661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.481243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.481990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.533160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.533331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.583823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.584059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.584150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.584323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.651725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.651963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.652956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.653088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.676426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.676622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.676738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.676860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.676963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.677944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.689571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.689779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.702595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.702842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703354 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.703919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.704002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.704135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.704349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:43.723440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.723654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:43.724008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.724177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.724259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:43.724332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:43.724489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.724590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.724752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.724834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:43.725014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.725096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.725271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.725361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.725526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.725777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.736289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.736587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.736742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.736856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.737023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.737137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.737293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.737582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.743954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.744467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.749868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.750153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.750237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.750436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.750519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.750717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.750799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.750997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.751088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.751259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.751342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.751515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.751594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.751778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.751860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.752048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.752129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.752312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.752466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.752607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.752689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.752868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.775364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.775460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.776878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.776968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.777157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.777239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.781745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.781846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.782024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.782108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.782305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.782387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.788881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.788989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.789120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.789203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.793249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.793474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.793571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.794274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.794368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.794562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.794646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.794840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.794925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.795110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.795294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.795374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.795445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.795640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.795723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.795917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.812205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.812509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.812608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.812788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.816405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.816672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.819522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.819778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.819862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.820075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.824189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.824419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.827375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.827582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.830557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.830729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.830859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.831022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.831148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.831302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.831438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.831581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.831737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.831855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.832027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.832138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.848460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.848557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.848736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.848819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.849001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.849084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.849261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.849345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.856047 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.856131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.858141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.858248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.858413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.858498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.861228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.861357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.861510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.861665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.861797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.861884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.862088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.862172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.862378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.862462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.873728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.873826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.874008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.874106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.874291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.874374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.874568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.874652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.874847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.874933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.875122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.875205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.875396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.875477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.890751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.890848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.893702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.893791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.896634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.896733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.899037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.899147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.900629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.900716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.900912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.900996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.901191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.901275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.908058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.908283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.909660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.909900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.912562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.912823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.912906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.913115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.913246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.913392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.913476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.913674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.921966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.922222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.922306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.922505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.922588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.922787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.929293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.929566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.929650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:43.929840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.929937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.930110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.930194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.930387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.930468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.930661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.930746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.930931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.931012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.931194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.931275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.931482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.931563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.931838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.931918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.932115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.932205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.932411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.932498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.932678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.932759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.932932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.933015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.933199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:43.966687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ebb5fdd9c0 pid: 80335 (default app) Jul 27 19:53:43.967026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ebb5fdd9c0 pid: 80336 (default app) Jul 27 19:53:43.969941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3ae43-7606-49c6-816d-35195b824840 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:43.970231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3ae43-7606-49c6-816d-35195b824840 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:43.970501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3ae43-7606-49c6-816d-35195b824840 None None] PATH_INFO: `/v3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:43.974782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Wed Jul 27 19:53:43 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:53:52.177052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1731a-c348-4c05-ac9a-fba918eef6f8 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:52.177600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1731a-c348-4c05-ac9a-fba918eef6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:52.177600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1731a-c348-4c05-ac9a-fba918eef6f8 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:52.182206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:53:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:53:52.187985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:52.188418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:52.188577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:52.258583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: INFO dbcounter [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] Registered counter for database keystone Jul 27 19:53:52.279999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG oslo_db.sqlalchemy.engines [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80335) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 19:53:52.286195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writer thread running {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 19:53:52.462475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG passlib.handlers.bcrypt [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80335) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 19:53:52.462813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG passlib.handlers.bcrypt [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80335) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 19:53:52.517101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:53:52.518703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:53:52.643053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:53:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 27 19:53:52.781658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1731a-c348-4c05-ac9a-fba918eef6f8 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:52.850644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: INFO dbcounter [None req-d7f1731a-c348-4c05-ac9a-fba918eef6f8 None None] Registered counter for database keystone Jul 27 19:53:52.869167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG oslo_db.sqlalchemy.engines [None req-d7f1731a-c348-4c05-ac9a-fba918eef6f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80336) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 19:53:52.875741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writer thread running {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 19:53:52.992982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1731a-c348-4c05-ac9a-fba918eef6f8 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:52.997892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8690cad-0acc-4e15-a1c0-dce86dd05e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:52.998535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690cad-0acc-4e15-a1c0-dce86dd05e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:52.998818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690cad-0acc-4e15-a1c0-dce86dd05e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:52.999095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690cad-0acc-4e15-a1c0-dce86dd05e5c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:53.007010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8690cad-0acc-4e15-a1c0-dce86dd05e5c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:53.060026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e8690cad-0acc-4e15-a1c0-dce86dd05e5c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 19:53:53.060307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8690cad-0acc-4e15-a1c0-dce86dd05e5c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:53.070016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Jul 27 19:53:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 19:53:53.076675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:53.091088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90469ac6-ce7c-48ee-966e-036935e48143 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:53.095644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d9181-c767-4d35-9081-31ab9352f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:53.096782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d9181-c767-4d35-9081-31ab9352f03e admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:53.097076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d9181-c767-4d35-9081-31ab9352f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:53.097366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d9181-c767-4d35-9081-31ab9352f03e admin admin] PATH_INFO: `/v3/projects/285f06e24b5441ccbaca698e1fbd1162` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:53.098850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d9181-c767-4d35-9081-31ab9352f03e admin admin] RBAC: Authorizing `identity:get_project(project_id=285f06e24b5441ccbaca698e1fbd1162)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:53.150678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING keystone.common.rbac_enforcer.enforcer [None req-549d9181-c767-4d35-9081-31ab9352f03e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 19:53:53.151031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d9181-c767-4d35-9081-31ab9352f03e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:53.155643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Wed Jul 27 19:53:53 2022] GET /identity/v3/projects/285f06e24b5441ccbaca698e1fbd1162 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:53:55.107396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6b7db-987a-4373-be3f-9e14f3267373 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:55.107958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6b7db-987a-4373-be3f-9e14f3267373 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:55.107958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6b7db-987a-4373-be3f-9e14f3267373 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:55.112058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:53:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:53:55.117439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb015fc-be7d-450f-8dbc-ae1345c73094 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:55.117816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb015fc-be7d-450f-8dbc-ae1345c73094 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:55.118048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb015fc-be7d-450f-8dbc-ae1345c73094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:55.137802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-0cb015fc-be7d-450f-8dbc-ae1345c73094 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:53:55.139407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0cb015fc-be7d-450f-8dbc-ae1345c73094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:53:55.147424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:53:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 27 19:53:55.169809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6b7db-987a-4373-be3f-9e14f3267373 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:55.181776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6b7db-987a-4373-be3f-9e14f3267373 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:55.186200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578b3a4-2999-4591-bbfc-6d68f411b71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:55.186839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578b3a4-2999-4591-bbfc-6d68f411b71f admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:55.187123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578b3a4-2999-4591-bbfc-6d68f411b71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:55.187397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578b3a4-2999-4591-bbfc-6d68f411b71f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:55.195004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578b3a4-2999-4591-bbfc-6d68f411b71f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:55.196794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578b3a4-2999-4591-bbfc-6d68f411b71f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:55.205564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Jul 27 19:53:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 19:53:55.210698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb015fc-be7d-450f-8dbc-ae1345c73094 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:55.220146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb015fc-be7d-450f-8dbc-ae1345c73094 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:55.224629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ded13-1f98-41fd-81ee-fac5e8c710fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:55.225263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ded13-1f98-41fd-81ee-fac5e8c710fa admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:55.225550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ded13-1f98-41fd-81ee-fac5e8c710fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:55.225828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ded13-1f98-41fd-81ee-fac5e8c710fa admin admin] PATH_INFO: `/v3/users/bf4d4f6d05b846a9b37142cd6195bb44` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:55.227206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ded13-1f98-41fd-81ee-fac5e8c710fa admin admin] RBAC: Authorizing `identity:get_user(user_id=bf4d4f6d05b846a9b37142cd6195bb44)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:55.229680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ded13-1f98-41fd-81ee-fac5e8c710fa admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:55.231225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Wed Jul 27 19:53:55 2022] GET /identity/v3/users/bf4d4f6d05b846a9b37142cd6195bb44 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:53:57.263697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7e736-bb62-47d7-a6d7-13485bffbf37 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.264255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7e736-bb62-47d7-a6d7-13485bffbf37 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.264255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7e736-bb62-47d7-a6d7-13485bffbf37 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.265617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:53:57.270609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ba7bf-f042-4b58-8259-1d3b5811aa08 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.271023 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ba7bf-f042-4b58-8259-1d3b5811aa08 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.271174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ba7bf-f042-4b58-8259-1d3b5811aa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.290059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-e35ba7bf-f042-4b58-8259-1d3b5811aa08 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:53:57.291757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e35ba7bf-f042-4b58-8259-1d3b5811aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:53:57.296181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7667ef-4c5c-4c87-ab4d-66255bcb7151 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.296498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7667ef-4c5c-4c87-ab4d-66255bcb7151 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.296780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7667ef-4c5c-4c87-ab4d-66255bcb7151 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.298108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:53:57.302956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.303244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.303524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.346298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG passlib.handlers.bcrypt [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80336) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 19:53:57.346735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG passlib.handlers.bcrypt [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80336) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 19:53:57.374435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:53:57.392097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:53:57.393728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:53:57.396654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ba7bf-f042-4b58-8259-1d3b5811aa08 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:57.421660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:53:57.500135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ba7bf-f042-4b58-8259-1d3b5811aa08 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:57.500679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:53:57.513295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:57.533424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:57.534005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.534291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.534571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.545829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:53:57.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:53:57.555604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:53:57.556516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:53:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:53:57.561858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:57.613300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797c90d-971a-4ec0-abb2-c5d1cc4f0049 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:57.615573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:53:57.641656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:53:57.648903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:57.649765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.650544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.650822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.659356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2966c5-4e3c-4905-9f1f-d3ea71db347b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:57.692726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:57.693345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.693633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.693912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.694634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:53:57.694943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:57.696343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:57.711121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:53:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:53:57.717446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:57.720891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:57.722424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:57.819197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:53:57.822923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99425ceb-664e-4e02-94d4-af4118260807 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:57.825226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:57.827912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:53:57.927910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f6760-9162-4a84-9fa4-f3b1a834b981 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:57.963397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:57.964040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.964252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:57.964409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.964611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] PATH_INFO: `/v3/users/bf4d4f6d05b846a9b37142cd6195bb44` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.965135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:57.965418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:57.965692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:57.965866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4d4f6d05b846a9b37142cd6195bb44)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:57.967325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:57.968952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:53:57 2022] GET /identity/v3/users/bf4d4f6d05b846a9b37142cd6195bb44 => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:53:57.974332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:57.984022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:57.985519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:57.988952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:53:58.003282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:53:58.004067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:53:58.004879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:53:58.004879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:53:58.004879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:53:58.004879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:53:58.004879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 9/18] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:53:58.010781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:58.066959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:53:58.075141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d54d61-005c-468b-83fa-9d5e41010a24 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:58.110969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:58.111615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:58.111895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:58.112169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:58.113426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:58.114867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:58.116297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd0ecd-e4b7-4089-93a8-0d10aa912b66 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:58.116463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 10/19] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:53:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:53:58.122779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:53:58.150379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd88544-594a-47d9-a7ec-e4530bbd2138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:58.151000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88544-594a-47d9-a7ec-e4530bbd2138 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:58.151284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88544-594a-47d9-a7ec-e4530bbd2138 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:58.151555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd88544-594a-47d9-a7ec-e4530bbd2138 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:58.152263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd88544-594a-47d9-a7ec-e4530bbd2138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:53:58.152594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd88544-594a-47d9-a7ec-e4530bbd2138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:58.153973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd88544-594a-47d9-a7ec-e4530bbd2138 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:58.171034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 10/20] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:53:58.207231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:53:58.226966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f8196-0877-43a9-90d8-0e673c052c64 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:53:58.264244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967389e7-0bc1-4a4a-8b1e-fe0f4bf629d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:53:58.264911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967389e7-0bc1-4a4a-8b1e-fe0f4bf629d1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:58.265195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967389e7-0bc1-4a4a-8b1e-fe0f4bf629d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:58.265471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967389e7-0bc1-4a4a-8b1e-fe0f4bf629d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:58.266288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967389e7-0bc1-4a4a-8b1e-fe0f4bf629d1 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.domain.id=default, user.id=bf4d4f6d05b846a9b37142cd6195bb44) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:53:58.266599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967389e7-0bc1-4a4a-8b1e-fe0f4bf629d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:53:58.268082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967389e7-0bc1-4a4a-8b1e-fe0f4bf629d1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:53:58.275522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 11/21] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Jul 27 19:53:58 2022] GET /identity/v3/role_assignments?user.id=bf4d4f6d05b846a9b37142cd6195bb44&scope.domain.id=default&role.id=7912a7addff34d9582eb2933d7110ebc => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:00.575940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7fb7-a3c1-4b5c-99f4-1ad3c10a1100 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:00.576978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7fb7-a3c1-4b5c-99f4-1ad3c10a1100 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:00.577453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d7fb7-a3c1-4b5c-99f4-1ad3c10a1100 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:00.578990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 11/22] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:00.584277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421c8ae-21ca-4e7d-b88f-ceffa8991f39 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:00.585138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421c8ae-21ca-4e7d-b88f-ceffa8991f39 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:00.585612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421c8ae-21ca-4e7d-b88f-ceffa8991f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:00.610612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3421c8ae-21ca-4e7d-b88f-ceffa8991f39 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:00.612111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3421c8ae-21ca-4e7d-b88f-ceffa8991f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:00.685183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453d498-4abe-44b2-b8f7-b93eb9915dd1 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:00.685878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453d498-4abe-44b2-b8f7-b93eb9915dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:00.686346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453d498-4abe-44b2-b8f7-b93eb9915dd1 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:00.687869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 12/23] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:00.697384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfd092-f399-4202-bc97-fe75e9727bfe None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:00.697911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfd092-f399-4202-bc97-fe75e9727bfe None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:00.698401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfd092-f399-4202-bc97-fe75e9727bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:00.710909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 12/24] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:00.719455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7cbfd092-f399-4202-bc97-fe75e9727bfe None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:00.720907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7cbfd092-f399-4202-bc97-fe75e9727bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:00.745490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421c8ae-21ca-4e7d-b88f-ceffa8991f39 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:00.843465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 13/25] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:00.877884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfd092-f399-4202-bc97-fe75e9727bfe None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:00.937575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:00.964758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:01.005869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421c8ae-21ca-4e7d-b88f-ceffa8991f39 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:01.059314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfd092-f399-4202-bc97-fe75e9727bfe None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:01.076802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616b98b-9dec-4897-b2ed-619d2f22a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:01.077445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b98b-9dec-4897-b2ed-619d2f22a60d None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.077782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b98b-9dec-4897-b2ed-619d2f22a60d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.078070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616b98b-9dec-4897-b2ed-619d2f22a60d None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.078812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b98b-9dec-4897-b2ed-619d2f22a60d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:01.080229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616b98b-9dec-4897-b2ed-619d2f22a60d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:01.112850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d34bf1-919e-4459-a936-600fb43266f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:01.113515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d34bf1-919e-4459-a936-600fb43266f6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.113803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d34bf1-919e-4459-a936-600fb43266f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.114084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d34bf1-919e-4459-a936-600fb43266f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.127144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d34bf1-919e-4459-a936-600fb43266f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-d5d34bf1-919e-4459-a936-600fb43266f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:01.133625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:01.134952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:01.135780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:01.139013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 14/26] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:01.140768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc162c3-2396-4116-9c8e-7806ea39d78e None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.141089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc162c3-2396-4116-9c8e-7806ea39d78e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.141453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc162c3-2396-4116-9c8e-7806ea39d78e None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.142954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 15/27] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:01.143830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1dc99-05da-4aea-bb9d-3fde0885b5ea None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.144121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1dc99-05da-4aea-bb9d-3fde0885b5ea None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.144414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1dc99-05da-4aea-bb9d-3fde0885b5ea None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.145764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 16/28] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:01.146455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1dc99-05da-4aea-bb9d-3fde0885b5ea None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:01.264612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1dc99-05da-4aea-bb9d-3fde0885b5ea None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:01.272214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:01.298665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96cbf4-a2d8-4a25-9e1b-8a1f628c3292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:01.299309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96cbf4-a2d8-4a25-9e1b-8a1f628c3292 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.299571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96cbf4-a2d8-4a25-9e1b-8a1f628c3292 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.299850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96cbf4-a2d8-4a25-9e1b-8a1f628c3292 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.300610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96cbf4-a2d8-4a25-9e1b-8a1f628c3292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:01.300920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96cbf4-a2d8-4a25-9e1b-8a1f628c3292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:01.302300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96cbf4-a2d8-4a25-9e1b-8a1f628c3292 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:01.338207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 13/29] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 27 19:54:00 2022] POST /identity/v3/roles => generated 219 bytes in 593 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:54:01.340673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1f17-0ee1-4857-a9b2-b66be724e437 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.341264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1f17-0ee1-4857-a9b2-b66be724e437 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.341749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f1f17-0ee1-4857-a9b2-b66be724e437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.364890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7f2f1f17-0ee1-4857-a9b2-b66be724e437 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:01.367115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7f2f1f17-0ee1-4857-a9b2-b66be724e437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:01.438782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 17/30] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:01.440078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3c0f2-4b51-4e5c-8879-f806cd4775dd None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.440374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3c0f2-4b51-4e5c-8879-f806cd4775dd None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.444086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3c0f2-4b51-4e5c-8879-f806cd4775dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.455076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 14/31] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:01.462285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f1f17-0ee1-4857-a9b2-b66be724e437 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:01.493742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-09c3c0f2-4b51-4e5c-8879-f806cd4775dd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:01.495108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-09c3c0f2-4b51-4e5c-8879-f806cd4775dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:01.554926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:01.591208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 18/32] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:01.592053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3c0f2-4b51-4e5c-8879-f806cd4775dd None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:01.599010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f1f17-0ee1-4857-a9b2-b66be724e437 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:01.637691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:01.639532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.640027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.640546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] PATH_INFO: `/v3/users/bf4d4f6d05b846a9b37142cd6195bb44` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.642103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] RBAC: Authorizing `identity:get_user(user_id=bf4d4f6d05b846a9b37142cd6195bb44)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:01.644208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:01.646375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 15/33] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:01 2022] GET /identity/v3/users/bf4d4f6d05b846a9b37142cd6195bb44 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:01.648587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:01.725175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:01.731571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3c0f2-4b51-4e5c-8879-f806cd4775dd None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:01.762850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529d685-18a6-446d-b5a5-ff5ade0fd60c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:01.771950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:01.772618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.772905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.773181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.773901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:01.775435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:01.797909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:01.803509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:01.804475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.805017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.806046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.818284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:01.820229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:54:01.835833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:01.837605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:01.838593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:54:01.838593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:54:01.838593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:54:01.838593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:01.839999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 16/34] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:01.841015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:01.950828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65528f29-a26e-4f2f-aa43-21a8c55bc871 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:01.980140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 19/35] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 27 19:54:01 2022] POST /identity/v3/roles => generated 225 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:01.981297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:01.996413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2ae00-cf43-40d1-a988-f5526bdc742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:01.996413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ae00-cf43-40d1-a988-f5526bdc742b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:01.996413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ae00-cf43-40d1-a988-f5526bdc742b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:01.996764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ae00-cf43-40d1-a988-f5526bdc742b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:01.998573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2ae00-cf43-40d1-a988-f5526bdc742b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:02.000908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2ae00-cf43-40d1-a988-f5526bdc742b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:02.002997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 17/36] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:02.005238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0155b-7543-4002-817c-4c3d710ad9b0 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.005529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0155b-7543-4002-817c-4c3d710ad9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.005813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0155b-7543-4002-817c-4c3d710ad9b0 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.007807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 18/37] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:02.008569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0155b-7543-4002-817c-4c3d710ad9b0 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:02.089249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:02.109547 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3a65-fa1e-4f91-b64f-7b48693d9a88 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:02.151559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300de32e-cdc9-4fff-ac38-07f79ff14e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:02.152204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300de32e-cdc9-4fff-ac38-07f79ff14e5f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.152558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300de32e-cdc9-4fff-ac38-07f79ff14e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.153368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300de32e-cdc9-4fff-ac38-07f79ff14e5f None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.153632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300de32e-cdc9-4fff-ac38-07f79ff14e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:02.153942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300de32e-cdc9-4fff-ac38-07f79ff14e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:02.155466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300de32e-cdc9-4fff-ac38-07f79ff14e5f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:02.159579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0155b-7543-4002-817c-4c3d710ad9b0 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:02.168222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 20/38] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:02.169602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bef61-25de-4fda-b19d-890bc72586fc None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.169911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bef61-25de-4fda-b19d-890bc72586fc None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.170175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bef61-25de-4fda-b19d-890bc72586fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.188798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-499bef61-25de-4fda-b19d-890bc72586fc None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:02.190331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-499bef61-25de-4fda-b19d-890bc72586fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:02.199133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffed48-2cb6-4ee6-a240-b9654e91874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:02.199820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffed48-2cb6-4ee6-a240-b9654e91874f None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.200103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffed48-2cb6-4ee6-a240-b9654e91874f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.200413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffed48-2cb6-4ee6-a240-b9654e91874f None admin] PATH_INFO: `/v3/domains/default/users/bf4d4f6d05b846a9b37142cd6195bb44/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.202456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffed48-2cb6-4ee6-a240-b9654e91874f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bf4d4f6d05b846a9b37142cd6195bb44, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:02.203915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffed48-2cb6-4ee6-a240-b9654e91874f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:02.205340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:02.259352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 21/39] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:02.260724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13bc2-3b16-44e8-a8a8-77460f45c969 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.261018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13bc2-3b16-44e8-a8a8-77460f45c969 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.261301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13bc2-3b16-44e8-a8a8-77460f45c969 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.262567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 22/40] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:02.263330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13bc2-3b16-44e8-a8a8-77460f45c969 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:02.283091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 19/41] 10.222.0.47 () {58 vars in 1361 bytes} [Wed Jul 27 19:54:02 2022] PUT /identity/v3/domains/default/users/bf4d4f6d05b846a9b37142cd6195bb44/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:02.287079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d930-29f7-462d-b9c3-3ce9044e0613 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.287379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d930-29f7-462d-b9c3-3ce9044e0613 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.287665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95d930-29f7-462d-b9c3-3ce9044e0613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.306839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fc95d930-29f7-462d-b9c3-3ce9044e0613 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:02.308678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fc95d930-29f7-462d-b9c3-3ce9044e0613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:02.385611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13bc2-3b16-44e8-a8a8-77460f45c969 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:02.388909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 20/42] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:02.389100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95d930-29f7-462d-b9c3-3ce9044e0613 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:02.443065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:02.445089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.445089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.445089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:02.447828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:02.449904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:02.506462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95d930-29f7-462d-b9c3-3ce9044e0613 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:02.540502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:02.541077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.541361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.541637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.542811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:02.544269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:02.545890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 21/43] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:02.546625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:02.570321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:02.584424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 23/44] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:54:02 2022] POST /identity/v3/projects => generated 301 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:02.584424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:02.673483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36117176-1896-4a4a-a49c-84ee9292cb2d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:02.690862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:02.724113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f475c-e9f9-4a6e-a5bb-5cb24d52b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:02.724804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f475c-e9f9-4a6e-a5bb-5cb24d52b196 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.725140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f475c-e9f9-4a6e-a5bb-5cb24d52b196 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.725423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f475c-e9f9-4a6e-a5bb-5cb24d52b196 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.726466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f475c-e9f9-4a6e-a5bb-5cb24d52b196 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:02.728238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f475c-e9f9-4a6e-a5bb-5cb24d52b196 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:02.731856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8abc2-b6ca-4a3c-bb05-15c09878ba86 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:02.776255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9db71-90d0-4cf6-b4b0-385df01216a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:02.776944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9db71-90d0-4cf6-b4b0-385df01216a1 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:02.777228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9db71-90d0-4cf6-b4b0-385df01216a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:02.777529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9db71-90d0-4cf6-b4b0-385df01216a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:02.778223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9db71-90d0-4cf6-b4b0-385df01216a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:02.779728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9db71-90d0-4cf6-b4b0-385df01216a1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:02.801998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 22/45] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 27 19:54:02 2022] POST /identity/v3/roles => generated 223 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:02.868877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 24/46] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:54:02 2022] POST /identity/v3/projects => generated 312 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:04.301049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6f0ee-c129-4a39-b43f-7c4834a125cf None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:04.301557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6f0ee-c129-4a39-b43f-7c4834a125cf None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:04.301639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6f0ee-c129-4a39-b43f-7c4834a125cf None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:04.302976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 23/47] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:04.308892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17a2c-c5ef-4b05-839e-6dda2afafaf3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:04.309303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17a2c-c5ef-4b05-839e-6dda2afafaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:04.309448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17a2c-c5ef-4b05-839e-6dda2afafaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:04.327426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d4a17a2c-c5ef-4b05-839e-6dda2afafaf3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:04.328637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d4a17a2c-c5ef-4b05-839e-6dda2afafaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:04.374520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:04.393664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 25/48] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:04.416247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6f0ee-c129-4a39-b43f-7c4834a125cf None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:04.518565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6f0ee-c129-4a39-b43f-7c4834a125cf None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:04.550950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:04.551637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:04.551873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:04.552153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:04.563086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:04.568533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:04.569495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:04.570422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 24/49] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:04.577083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a17a2c-c5ef-4b05-839e-6dda2afafaf3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:04.673235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a17a2c-c5ef-4b05-839e-6dda2afafaf3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:04.705965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:04.706583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:04.706869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:04.707148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:04.707864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:04.708175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:04.709603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:04.717794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 26/50] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:04.723279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:04.754284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:04.823017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4cdc1-ed98-4431-b7ab-8fd4032b4d86 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:04.857853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7a14-fa65-4432-b3cb-decca3febf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:04.858495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7a14-fa65-4432-b3cb-decca3febf88 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:04.858786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7a14-fa65-4432-b3cb-decca3febf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:04.859061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7a14-fa65-4432-b3cb-decca3febf88 None admin] PATH_INFO: `/v3/users/bf4d4f6d05b846a9b37142cd6195bb44` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:04.860239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7a14-fa65-4432-b3cb-decca3febf88 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4d4f6d05b846a9b37142cd6195bb44)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:04.861725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7a14-fa65-4432-b3cb-decca3febf88 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:04.863365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 25/51] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:04 2022] GET /identity/v3/users/bf4d4f6d05b846a9b37142cd6195bb44 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:04.868686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:04.879864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad64b6-26bf-44e0-917b-4e53cc5c1cc1 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:04.880154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad64b6-26bf-44e0-917b-4e53cc5c1cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:04.880472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad64b6-26bf-44e0-917b-4e53cc5c1cc1 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:04.881787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 26/52] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:04.886462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44b92-ebc0-460f-862a-7b6373821a89 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:04.886750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44b92-ebc0-460f-862a-7b6373821a89 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:04.887028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca44b92-ebc0-460f-862a-7b6373821a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:04.905529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3ca44b92-ebc0-460f-862a-7b6373821a89 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:04.906767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3ca44b92-ebc0-460f-862a-7b6373821a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:04.915819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:04.970659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7f4ef-8b89-44b0-8d42-894f9b710316 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:04.972804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 27/53] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:04.995602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44b92-ebc0-460f-862a-7b6373821a89 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:05.003970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:05.004651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:05.004937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:05.005221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:05.006367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:05.008000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:05.009622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 27/54] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:05.015110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:05.027154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:05.099026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca44b92-ebc0-460f-862a-7b6373821a89 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:05.114136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fa169-710e-4cf4-b255-05e1ea4a1522 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:05.126817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:05.132901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:05.133607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:05.133895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:05.134208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:05.135381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:05.136858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:05.138572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 28/55] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:05.144247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:05.147188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36ec5-6f05-456c-97de-1d5916bb0fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:05.147824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36ec5-6f05-456c-97de-1d5916bb0fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:05.148112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36ec5-6f05-456c-97de-1d5916bb0fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:05.148940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36ec5-6f05-456c-97de-1d5916bb0fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:05.149737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36ec5-6f05-456c-97de-1d5916bb0fa2 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.domain.id=default, user.id=bf4d4f6d05b846a9b37142cd6195bb44) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:05.150082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36ec5-6f05-456c-97de-1d5916bb0fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:05.151521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d36ec5-6f05-456c-97de-1d5916bb0fa2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:05.160742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 28/56] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Jul 27 19:54:05 2022] GET /identity/v3/role_assignments?user.id=bf4d4f6d05b846a9b37142cd6195bb44&scope.domain.id=default&role.id=7912a7addff34d9582eb2933d7110ebc => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:05.245158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:05.251141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5bdf9-2df9-4cb7-8524-c3e084fbac63 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:05.287518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e8e6a-5d04-4003-b09f-709bea148956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:05.288154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e8e6a-5d04-4003-b09f-709bea148956 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:05.288475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e8e6a-5d04-4003-b09f-709bea148956 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:05.288758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e8e6a-5d04-4003-b09f-709bea148956 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:05.289466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e8e6a-5d04-4003-b09f-709bea148956 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:05.290948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e8e6a-5d04-4003-b09f-709bea148956 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:05.484090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 29/57] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Jul 27 19:54:05 2022] POST /identity/v3/projects => generated 298 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:54:07.461921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a390d-1fb3-462c-a6d8-abc26d0a7c3f None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:07.462405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a390d-1fb3-462c-a6d8-abc26d0a7c3f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:07.462483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a390d-1fb3-462c-a6d8-abc26d0a7c3f None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:07.463837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 29/58] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:07.469073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11565a-e203-46d0-b237-0b2f3064a68e None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:07.469453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11565a-e203-46d0-b237-0b2f3064a68e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:07.469640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11565a-e203-46d0-b237-0b2f3064a68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:07.487897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-bd11565a-e203-46d0-b237-0b2f3064a68e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:07.489125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bd11565a-e203-46d0-b237-0b2f3064a68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:07.553506 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 30/59] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:07.577323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a390d-1fb3-462c-a6d8-abc26d0a7c3f None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:07.674161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a390d-1fb3-462c-a6d8-abc26d0a7c3f None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:07.706725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd511792-2438-433f-b83f-9d39a1e4a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:07.707351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd511792-2438-433f-b83f-9d39a1e4a02a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:07.707636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd511792-2438-433f-b83f-9d39a1e4a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:07.707917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd511792-2438-433f-b83f-9d39a1e4a02a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:07.709089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd511792-2438-433f-b83f-9d39a1e4a02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:07.710564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd511792-2438-433f-b83f-9d39a1e4a02a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:07.712130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 30/60] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:07.717408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11565a-e203-46d0-b237-0b2f3064a68e None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:07.792952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:07.813594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11565a-e203-46d0-b237-0b2f3064a68e None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:07.845629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5698b3b-d682-4321-bd54-04d27ba2fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:07.846237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5698b3b-d682-4321-bd54-04d27ba2fa38 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:07.846518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5698b3b-d682-4321-bd54-04d27ba2fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:07.846858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5698b3b-d682-4321-bd54-04d27ba2fa38 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:07.847529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5698b3b-d682-4321-bd54-04d27ba2fa38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:07.849044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5698b3b-d682-4321-bd54-04d27ba2fa38 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:07.923219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 31/61] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:07 2022] POST /identity/v3/users => generated 275 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:10.844070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329ef03-3708-4524-ad33-1e4e01413ca9 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:10.844624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329ef03-3708-4524-ad33-1e4e01413ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:10.844727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329ef03-3708-4524-ad33-1e4e01413ca9 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:10.846009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 31/62] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:10.860703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674626d9-c4b1-43aa-a2b4-57ba1e8a3449 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:10.861512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674626d9-c4b1-43aa-a2b4-57ba1e8a3449 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:10.861972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674626d9-c4b1-43aa-a2b4-57ba1e8a3449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:10.889332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-674626d9-c4b1-43aa-a2b4-57ba1e8a3449 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:10.890786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-674626d9-c4b1-43aa-a2b4-57ba1e8a3449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:10.962339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 32/63] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:11.009804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329ef03-3708-4524-ad33-1e4e01413ca9 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.066725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:11.083532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b4203-327a-454c-8937-fbaa7ebf94a0 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.084117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b4203-327a-454c-8937-fbaa7ebf94a0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.084703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b4203-327a-454c-8937-fbaa7ebf94a0 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.088149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 33/64] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:11.106214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd56093-adfa-4411-8fd6-88bf3cd377e1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.106457 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd56093-adfa-4411-8fd6-88bf3cd377e1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.106713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd56093-adfa-4411-8fd6-88bf3cd377e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.131271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fdd56093-adfa-4411-8fd6-88bf3cd377e1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:11.133048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fdd56093-adfa-4411-8fd6-88bf3cd377e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:11.151763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329ef03-3708-4524-ad33-1e4e01413ca9 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.221866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:11.232021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 34/65] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:11.234088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.238955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.238955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.238955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.246890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:11.251996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:11.252999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:11.253745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 32/66] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:11.259294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd56093-adfa-4411-8fd6-88bf3cd377e1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.275338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.395752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd56093-adfa-4411-8fd6-88bf3cd377e1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.414264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5528821-8d39-417e-a3a9-ca1136aed944 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.417856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:11.444278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3e494-699f-4483-97da-e8650dde870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.444964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e494-699f-4483-97da-e8650dde870e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.445531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e494-699f-4483-97da-e8650dde870e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.445851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3e494-699f-4483-97da-e8650dde870e None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.446602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3e494-699f-4483-97da-e8650dde870e None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:11.446917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3e494-699f-4483-97da-e8650dde870e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:11.449893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3e494-699f-4483-97da-e8650dde870e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:11.453836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf49093-a6b4-4753-b7e3-5b50ede2491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.454700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf49093-a6b4-4753-b7e3-5b50ede2491a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.455161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf49093-a6b4-4753-b7e3-5b50ede2491a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.455631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf49093-a6b4-4753-b7e3-5b50ede2491a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.459639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 35/67] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:11.460942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810acc13-62f2-4899-b41f-ff7768a6a663 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.461488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810acc13-62f2-4899-b41f-ff7768a6a663 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.461786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810acc13-62f2-4899-b41f-ff7768a6a663 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.463029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 36/68] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:11.465220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810acc13-62f2-4899-b41f-ff7768a6a663 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.473284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf49093-a6b4-4753-b7e3-5b50ede2491a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-5cf49093-a6b4-4753-b7e3-5b50ede2491a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:11.480185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:11.481256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:11.482260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:11.484931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 33/69] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:11.486496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825cd83-5da7-428d-8e01-210524e3d3ce None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.486972 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825cd83-5da7-428d-8e01-210524e3d3ce None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.487380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825cd83-5da7-428d-8e01-210524e3d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.511528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e825cd83-5da7-428d-8e01-210524e3d3ce None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:11.513002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e825cd83-5da7-428d-8e01-210524e3d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:11.536903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:11.593723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 34/70] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:11.594849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825cd83-5da7-428d-8e01-210524e3d3ce None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.611118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810acc13-62f2-4899-b41f-ff7768a6a663 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.651747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.652779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.653244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.653696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.669389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:11.670868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:11.672779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 37/71] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:11.673378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.718568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:11.724709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825cd83-5da7-428d-8e01-210524e3d3ce None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.757579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c637-9e4f-4ee0-bc86-3d812abafa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.758199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c637-9e4f-4ee0-bc86-3d812abafa4e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.758483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c637-9e4f-4ee0-bc86-3d812abafa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.758758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c637-9e4f-4ee0-bc86-3d812abafa4e None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.759470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26c637-9e4f-4ee0-bc86-3d812abafa4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:11.759773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26c637-9e4f-4ee0-bc86-3d812abafa4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:11.761167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26c637-9e4f-4ee0-bc86-3d812abafa4e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:11.769443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 35/72] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:11.769443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bf61f-a8a7-44f0-b669-4562daa8790a None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.769443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bf61f-a8a7-44f0-b669-4562daa8790a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.769443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bf61f-a8a7-44f0-b669-4562daa8790a None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.772891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 36/73] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:11.773703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bf61f-a8a7-44f0-b669-4562daa8790a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.777158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f1da-4673-4369-98e0-777cf032443d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.791627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:11.811352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e9693-3306-4172-9064-0b135e3370ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.811992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e9693-3306-4172-9064-0b135e3370ef None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.812278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e9693-3306-4172-9064-0b135e3370ef None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.812584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e9693-3306-4172-9064-0b135e3370ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.822608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e9693-3306-4172-9064-0b135e3370ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-0e3e9693-3306-4172-9064-0b135e3370ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:11.827649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:11.828566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:11.829346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 38/74] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:11.831277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1382a9-6fab-45b5-8831-612e508a2ba7 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.831568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1382a9-6fab-45b5-8831-612e508a2ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.831845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1382a9-6fab-45b5-8831-612e508a2ba7 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.833272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 39/75] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:11.843638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1382a9-6fab-45b5-8831-612e508a2ba7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:11.883113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bf61f-a8a7-44f0-b669-4562daa8790a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.916221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c713a-e013-4813-bc04-b14099b14720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.916811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c713a-e013-4813-bc04-b14099b14720 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.917095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c713a-e013-4813-bc04-b14099b14720 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.917368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c713a-e013-4813-bc04-b14099b14720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.941132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:11.942689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1382a9-6fab-45b5-8831-612e508a2ba7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:11.977838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6ff81-ce79-4232-9fdf-457c9407020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:11.978488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ff81-ce79-4232-9fdf-457c9407020c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.978776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ff81-ce79-4232-9fdf-457c9407020c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.979062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ff81-ce79-4232-9fdf-457c9407020c None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.980255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6ff81-ce79-4232-9fdf-457c9407020c None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:11.981751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6ff81-ce79-4232-9fdf-457c9407020c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:11.983376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 40/76] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:11.984512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fc663-9621-4ca3-a74d-61aad74532f7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:11.984800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fc663-9621-4ca3-a74d-61aad74532f7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:11.985082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fc663-9621-4ca3-a74d-61aad74532f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:11.989166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c713a-e013-4813-bc04-b14099b14720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:11.990680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c713a-e013-4813-bc04-b14099b14720 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.002263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b22fc663-9621-4ca3-a74d-61aad74532f7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:12.003481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b22fc663-9621-4ca3-a74d-61aad74532f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:12.039023 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.070055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 41/77] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:12.077259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fc663-9621-4ca3-a74d-61aad74532f7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.090574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 37/78] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:12.091929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276596-9fd0-4dd2-b0db-0c646a38b802 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.092317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276596-9fd0-4dd2-b0db-0c646a38b802 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.092717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276596-9fd0-4dd2-b0db-0c646a38b802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.110092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e6276596-9fd0-4dd2-b0db-0c646a38b802 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:12.111258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e6276596-9fd0-4dd2-b0db-0c646a38b802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:12.160584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.177115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 38/79] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:12.177957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276596-9fd0-4dd2-b0db-0c646a38b802 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.178278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fc663-9621-4ca3-a74d-61aad74532f7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.213416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.214043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.214326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.214603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.215324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:12.215632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.217042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.224937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 42/80] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:12.225716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.258608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.290047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276596-9fd0-4dd2-b0db-0c646a38b802 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.323393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.324013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.324160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3478c05-67a9-4af8-91fb-e20cab60f5f3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.324295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.324597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.358337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.358982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.359267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.359568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.360747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.362192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.363959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 43/81] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:12.364656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.383198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.395034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.396546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.463932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db365fd5-8c48-49ff-bc7d-3c7fe8cea75b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.465388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.494206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 39/82] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:12.497006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.497489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.497671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.497959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.498243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.499425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.500915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.502580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 44/83] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:12.503400 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.581819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.596401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252fe4e-7674-4358-93b9-111e302d620a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.601852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411937a6-4210-40de-a48e-3fb4cf7e3c9d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.630038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.630680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.630963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.631761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.634931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.635555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.635836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.636112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] PATH_INFO: `/v3/users/bf4d4f6d05b846a9b37142cd6195bb44` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.637291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4d4f6d05b846a9b37142cd6195bb44)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.638742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.640322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 45/84] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/users/bf4d4f6d05b846a9b37142cd6195bb44 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:12.641074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.642118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:12.647200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:12.648084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:12.648860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 40/85] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:12.649469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.686404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.745477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4fa8d-a718-4725-9f0e-f6d603c1b537 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.746959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c815c1-38c0-46fa-bc51-ef8210172951 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.777740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.778349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.778608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.778732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.778914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.779235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.779515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.779595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.779791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.780974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.781070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.782408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.784170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 41/86] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:12.789772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.800898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.882309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e65605-4129-4bf2-bed4-6f8d73c1dce3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.886915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 46/87] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:12 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:12.887844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.922081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.923269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.923559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.923833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.924842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=a3dfb349996e4ef189c4b08a2a09941e) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:12.925153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:12.926634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:12.933465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 42/88] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/role_assignments?user.id=a3dfb349996e4ef189c4b08a2a09941e&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:12.934329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.024813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.034429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f541d-a212-44dc-b704-d10fc2650209 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.056002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.087415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261082c-556c-4166-88a7-ce926e7c6fc1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.089060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.089944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.091230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.091524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.134526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.135167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.135450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.135766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.136498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:13.136807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.138213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.151202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 43/89] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:13.152357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.205374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.207896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.298793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01cf60-b5d6-4667-96d1-b5bee3c38e3c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.325405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.333162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 47/90] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:13.334830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.346675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.347325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.347619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.347901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.348732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] RBAC: Adding query filter params (role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=a3dfb349996e4ef189c4b08a2a09941e) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:13.349047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.351247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.358007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 44/91] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/role_assignments?user.id=a3dfb349996e4ef189c4b08a2a09941e&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:13.365270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.369168 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.457425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0454f-408b-4bcd-830f-8ed68b7a3876 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.499045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.500068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.500677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.501443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.503564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.505544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.509494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 48/92] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:13.517645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.576475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74bb6-a8f6-4352-a69a-8663520540a6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.640615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.641846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.643104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.643596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.648751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.649041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.650801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.653957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 45/93] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:13.659585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.690645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce8fb6-49ca-43d4-9e2a-abe6a9716634 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.753266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.753961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.754247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.754579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.792789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.844023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62447371-fb1d-496e-a760-561300ad2d56 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.886128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.887623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.924515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.925207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.925502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.925785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.926601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=bf4d4f6d05b846a9b37142cd6195bb44) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:13.926919 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.928419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.944606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 46/94] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/role_assignments?user.id=bf4d4f6d05b846a9b37142cd6195bb44&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:14.014088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:14.066448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 49/95] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:14.071769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:14.204990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ca4ac-4b42-4793-b839-03f50752afde None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:14.258795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e7aa6-de8e-41f6-ba07-1e545752621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:14.259477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7aa6-de8e-41f6-ba07-1e545752621f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:14.259763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7aa6-de8e-41f6-ba07-1e545752621f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:14.260042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e7aa6-de8e-41f6-ba07-1e545752621f None admin] PATH_INFO: `/v3/projects/d2e6e396c514416296a551d5165c2d6b` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:14.265116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e7aa6-de8e-41f6-ba07-1e545752621f None admin] RBAC: Authorizing `identity:get_project(project_id=d2e6e396c514416296a551d5165c2d6b)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:14.266602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e7aa6-de8e-41f6-ba07-1e545752621f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:14.268509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 47/96] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:14 2022] GET /identity/v3/projects/d2e6e396c514416296a551d5165c2d6b => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:14.274812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:14.398375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892a71c-292b-4fda-aaa4-c78543dde70c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:14.447007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cea69-3cd8-4405-8c9e-7306fd1dc033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:14.447641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cea69-3cd8-4405-8c9e-7306fd1dc033 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:14.447925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cea69-3cd8-4405-8c9e-7306fd1dc033 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:14.448206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cea69-3cd8-4405-8c9e-7306fd1dc033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:14.449045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cea69-3cd8-4405-8c9e-7306fd1dc033 None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=d2e6e396c514416296a551d5165c2d6b, user.id=a3dfb349996e4ef189c4b08a2a09941e) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:14.449357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cea69-3cd8-4405-8c9e-7306fd1dc033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:14.450815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cea69-3cd8-4405-8c9e-7306fd1dc033 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:14.461614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 50/97] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:14 2022] GET /identity/v3/role_assignments?user.id=a3dfb349996e4ef189c4b08a2a09941e&scope.project.id=d2e6e396c514416296a551d5165c2d6b&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:15.658419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c00c9-12bd-4d84-9686-710752f8f5d6 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.659245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c00c9-12bd-4d84-9686-710752f8f5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.659675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c00c9-12bd-4d84-9686-710752f8f5d6 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.661223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 48/98] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:15.666636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35afa-1b77-4e7c-92d6-ce4a5f7abe29 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.667368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35afa-1b77-4e7c-92d6-ce4a5f7abe29 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.667848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd35afa-1b77-4e7c-92d6-ce4a5f7abe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.695114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76836800-2d0f-40d5-88e9-168b3d40e0f8 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.695415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76836800-2d0f-40d5-88e9-168b3d40e0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.695677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76836800-2d0f-40d5-88e9-168b3d40e0f8 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.697179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 49/99] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:15.697311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-dfd35afa-1b77-4e7c-92d6-ce4a5f7abe29 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:15.698762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dfd35afa-1b77-4e7c-92d6-ce4a5f7abe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:15.704311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a842f1-8a5f-44d2-b9ae-95b19fbb962d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.704884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a842f1-8a5f-44d2-b9ae-95b19fbb962d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.705362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a842f1-8a5f-44d2-b9ae-95b19fbb962d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.716226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:15.732112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b8a842f1-8a5f-44d2-b9ae-95b19fbb962d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:15.734204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b8a842f1-8a5f-44d2-b9ae-95b19fbb962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:15.744748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:15.829572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 51/100] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:15.852904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 50/101] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 27 19:54:15.870860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35afa-1b77-4e7c-92d6-ce4a5f7abe29 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:15.884494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a842f1-8a5f-44d2-b9ae-95b19fbb962d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.043589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd35afa-1b77-4e7c-92d6-ce4a5f7abe29 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.060931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a842f1-8a5f-44d2-b9ae-95b19fbb962d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.081293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.081962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.082253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.082568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.101240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:16.103706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.104618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.105098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.105597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:16.106466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:16.107258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:16.112901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 52/102] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:16.115707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.117356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.125760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:16.131752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:16.132783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:16.133517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:16.135781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 51/103] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:16.141494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.158596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.272171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7355c-955c-401d-abb4-6872c4daccca None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.300039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b4642-1a17-4670-b9c2-184f8902e5c1 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.307474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4777e7-b9ab-4f57-a207-4dbcafeef8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.308331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4777e7-b9ab-4f57-a207-4dbcafeef8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.308954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4777e7-b9ab-4f57-a207-4dbcafeef8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.309426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4777e7-b9ab-4f57-a207-4dbcafeef8c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.310337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4777e7-b9ab-4f57-a207-4dbcafeef8c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:16.310837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4777e7-b9ab-4f57-a207-4dbcafeef8c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.312454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4777e7-b9ab-4f57-a207-4dbcafeef8c6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.321770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 52/104] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:16.323368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2b855-2a0d-48d2-b545-48e44a8df295 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.323871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2b855-2a0d-48d2-b545-48e44a8df295 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.324345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2b855-2a0d-48d2-b545-48e44a8df295 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.325779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 53/105] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:16.335238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38e783-5223-4cd1-8ea6-d150e68ac504 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.335802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38e783-5223-4cd1-8ea6-d150e68ac504 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.336332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38e783-5223-4cd1-8ea6-d150e68ac504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.355791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-af38e783-5223-4cd1-8ea6-d150e68ac504 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:16.357300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-af38e783-5223-4cd1-8ea6-d150e68ac504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:16.382645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.383312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.383551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.383828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.384673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:16.384884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.386335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.408499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 53/106] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:16.408499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.435344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 54/107] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:16.435772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.444816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38e783-5223-4cd1-8ea6-d150e68ac504 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.593296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.627178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb5ed8-1e8a-40e3-b712-7481ea63d05b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.629808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38e783-5223-4cd1-8ea6-d150e68ac504 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.665324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.665978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.666268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.666548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.667765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.668323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6e642-f772-4cf6-92f5-ab0c7bc615ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.668699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e642-f772-4cf6-92f5-ab0c7bc615ef None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.669057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e642-f772-4cf6-92f5-ab0c7bc615ef None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.669243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.669509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e642-f772-4cf6-92f5-ab0c7bc615ef None admin] PATH_INFO: `/v3/users/1e2f83a6a3604ec288302cbd0e6c866f` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.670869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 54/108] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:16.672614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.693928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6e642-f772-4cf6-92f5-ab0c7bc615ef None admin] RBAC: Authorizing `identity:get_user(user_id=1e2f83a6a3604ec288302cbd0e6c866f)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.696096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6e642-f772-4cf6-92f5-ab0c7bc615ef None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.698487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 55/109] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/users/1e2f83a6a3604ec288302cbd0e6c866f => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:16.699735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c04af-db12-4f44-bc1d-05576fafce69 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.700019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c04af-db12-4f44-bc1d-05576fafce69 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.700291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c04af-db12-4f44-bc1d-05576fafce69 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.702142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 56/110] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:16.704620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c04af-db12-4f44-bc1d-05576fafce69 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.755312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.792326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9795a9-b484-490a-add7-1f9aef19e43b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.813987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c04af-db12-4f44-bc1d-05576fafce69 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.829582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.830226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.830509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.830785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.832149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.834176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.835879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 55/111] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:16.836664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.853781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.856182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71a668-f303-47b1-a40b-0bcfb9d06e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.856926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71a668-f303-47b1-a40b-0bcfb9d06e3d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.857214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71a668-f303-47b1-a40b-0bcfb9d06e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.857563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71a668-f303-47b1-a40b-0bcfb9d06e3d None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.858784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71a668-f303-47b1-a40b-0bcfb9d06e3d None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.860351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71a668-f303-47b1-a40b-0bcfb9d06e3d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.862132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 57/112] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:16.868234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029f764-87cf-4202-bc83-13f197968df0 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.868440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029f764-87cf-4202-bc83-13f197968df0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.868731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029f764-87cf-4202-bc83-13f197968df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.888312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-0029f764-87cf-4202-bc83-13f197968df0 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:16.889601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0029f764-87cf-4202-bc83-13f197968df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:16.948521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdb06c-d6c4-41be-8d17-b0d5e7e99be2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.961445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 58/113] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:16.968531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029f764-87cf-4202-bc83-13f197968df0 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.004417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.005850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.006557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.007030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.018632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.078544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029f764-87cf-4202-bc83-13f197968df0 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.090073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.091555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.109042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.115690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.120617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.120617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.120617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.120617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.120617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.192913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 59/114] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:54:16 2022] POST /identity/v3/projects => generated 302 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:17.192913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.206571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 56/115] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:17.211706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.306643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.316076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaba8c-f145-43b8-9a9f-f2cd6ddb9617 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.361570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643791e8-792b-4903-a4e0-bb1ae3b56666 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.375862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66652a9a-861e-47d8-8a02-6bd2d6330d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.377768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66652a9a-861e-47d8-8a02-6bd2d6330d5e None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.377768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66652a9a-861e-47d8-8a02-6bd2d6330d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.377768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66652a9a-861e-47d8-8a02-6bd2d6330d5e None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861/users/a3dfb349996e4ef189c4b08a2a09941e/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.378755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.380477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66652a9a-861e-47d8-8a02-6bd2d6330d5e None admin] RBAC: Authorizing `identity:create_grant(project_id=3267824b193a4a3082969dc9c0836861, user_id=a3dfb349996e4ef189c4b08a2a09941e, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.381967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66652a9a-861e-47d8-8a02-6bd2d6330d5e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.402099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada6b55-9caa-40b8-94cb-67ab23c381ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.402716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada6b55-9caa-40b8-94cb-67ab23c381ce None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.403007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada6b55-9caa-40b8-94cb-67ab23c381ce None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.403381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada6b55-9caa-40b8-94cb-67ab23c381ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.414629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada6b55-9caa-40b8-94cb-67ab23c381ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-cada6b55-9caa-40b8-94cb-67ab23c381ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:17.420090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:17.421041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:17.422192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 57/116] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:17.422802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea12fa-1da4-4b28-90f4-0f832fcfa1d3 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.423078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea12fa-1da4-4b28-90f4-0f832fcfa1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.423541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea12fa-1da4-4b28-90f4-0f832fcfa1d3 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.425997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 58/117] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:17.427386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea12fa-1da4-4b28-90f4-0f832fcfa1d3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.485724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 60/118] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:17 2022] PUT /identity/v3/projects/3267824b193a4a3082969dc9c0836861/users/a3dfb349996e4ef189c4b08a2a09941e/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:17.487550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cab31-d1e3-401c-98cc-d730495d1155 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.487913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cab31-d1e3-401c-98cc-d730495d1155 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.488267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cab31-d1e3-401c-98cc-d730495d1155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.513629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bb5cab31-d1e3-401c-98cc-d730495d1155 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:17.515359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bb5cab31-d1e3-401c-98cc-d730495d1155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:17.562256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea12fa-1da4-4b28-90f4-0f832fcfa1d3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.603469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.604246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.604627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.604977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.606456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.608218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.610297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 59/119] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:17 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:17.613028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 61/120] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:17.614846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaaeac-289b-4c62-91be-d2c5aa760694 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.615137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaaeac-289b-4c62-91be-d2c5aa760694 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.615414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaaeac-289b-4c62-91be-d2c5aa760694 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.616907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 62/121] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:17.617267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.622073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaaeac-289b-4c62-91be-d2c5aa760694 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.684435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.736145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.761530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaaeac-289b-4c62-91be-d2c5aa760694 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.761981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab758921-b62e-4b26-a86d-e7fe45293298 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.800350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.801166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.801453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.802253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.803291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:17.803602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.805786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.811910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66899a3-bf95-43a9-b03e-9dfe59343104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.812609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66899a3-bf95-43a9-b03e-9dfe59343104 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.812896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66899a3-bf95-43a9-b03e-9dfe59343104 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.813174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66899a3-bf95-43a9-b03e-9dfe59343104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.813962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66899a3-bf95-43a9-b03e-9dfe59343104 None admin] RBAC: Adding query filter params (role.id=46fcb1e9e111430489c72b70f557e173, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=1e2f83a6a3604ec288302cbd0e6c866f) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:17.814325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66899a3-bf95-43a9-b03e-9dfe59343104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.815805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66899a3-bf95-43a9-b03e-9dfe59343104 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.817995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 60/122] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:17.822504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 63/123] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:17 2022] GET /identity/v3/role_assignments?user.id=1e2f83a6a3604ec288302cbd0e6c866f&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=46fcb1e9e111430489c72b70f557e173 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:17.823678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.823803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61906169-642e-4418-91b5-cc2d3a4c0cab None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.824016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61906169-642e-4418-91b5-cc2d3a4c0cab None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.824287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61906169-642e-4418-91b5-cc2d3a4c0cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.843764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-61906169-642e-4418-91b5-cc2d3a4c0cab None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:17.845237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-61906169-642e-4418-91b5-cc2d3a4c0cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:17.918843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 64/124] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:17.919645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61906169-642e-4418-91b5-cc2d3a4c0cab None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.950187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edcac58-b4ca-4857-bc8c-1ac402e23ee9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.953283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.991877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.992939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.993839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.994573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.007441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:18.014055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:18.014962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:18.015710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:18.017909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 61/125] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:18.018959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.022476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:18.048100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61906169-642e-4418-91b5-cc2d3a4c0cab None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.118656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.119544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.119970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.120432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.121830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.123427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.125290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 65/126] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:17 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:18.126255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.163257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ec565-a369-4dae-9d7c-d4a7c48d5847 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.253163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.254190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.254670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.255865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.279632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:18.290742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:18.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:18.292357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:18.296429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 62/127] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:18.299595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.334144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2711afe-f442-412a-b0b1-4e2695631de3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.356774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:18.414480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.415113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.415398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.415675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.416429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:18.416723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.418259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.428230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 66/128] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:18.429086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.453889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:18.499763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963879b8-b3ee-4477-a2b4-1c5f7836c168 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.593405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.593981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.594263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.594537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.595918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.597409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.609578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 63/129] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:18 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:18.610427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.650716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f69ce0-d083-4b8a-bb89-ee8219fa6c57 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.700294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.700967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.701256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.701541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.702276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:18.702584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.703972 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.723015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 67/130] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:18.723899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.757684 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:18.771586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905e884-9a60-45d1-9df1-5b56f104e555 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.835005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.835650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.835929 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.836201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] PATH_INFO: `/v3/users/bf4d4f6d05b846a9b37142cd6195bb44` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.838365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4d4f6d05b846a9b37142cd6195bb44)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.840594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.845820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 64/131] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:18 2022] GET /identity/v3/users/bf4d4f6d05b846a9b37142cd6195bb44 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:18.847524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.850029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:18.902133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be2dd6-f92d-4478-8e96-fed07563a4aa None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.951078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.952089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.952623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.953078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861/users/a3dfb349996e4ef189c4b08a2a09941e/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.955622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=3267824b193a4a3082969dc9c0836861, user_id=a3dfb349996e4ef189c4b08a2a09941e, role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.957276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:19.015086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40801f-f735-4910-865b-7381873acbb3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:19.051985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:19.052958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.053449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.053895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.055510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:19.057163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:19.058943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 65/132] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:18 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:19.060085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:19.079721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 68/133] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:18 2022] PUT /identity/v3/projects/3267824b193a4a3082969dc9c0836861/users/a3dfb349996e4ef189c4b08a2a09941e/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:19.080983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:19.100442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:19.177636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c49ee-7d8f-476b-bf36-c23830cfd08c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:19.214941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:19.215925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.216416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.216869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.218273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:19.219994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:19.221917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 66/134] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:19 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:19.228331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:19.320923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f537e-8284-461c-8921-bd3693427d23 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:19.392486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:19.400599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:19.401273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.401562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.401842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] PATH_INFO: `/v3/projects/d2e6e396c514416296a551d5165c2d6b` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.406074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] RBAC: Authorizing `identity:get_project(project_id=d2e6e396c514416296a551d5165c2d6b)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:19.406537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:19.408913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 69/135] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:19 2022] GET /identity/v3/projects/d2e6e396c514416296a551d5165c2d6b => generated 312 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:19.419440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:19.470497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:19.480020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae96d89-aa25-4232-87d2-a5eea8370acc None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:19.529761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c3d07-ca39-4c0d-95d5-e0885f7a2974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:19.530481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c3d07-ca39-4c0d-95d5-e0885f7a2974 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.530769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c3d07-ca39-4c0d-95d5-e0885f7a2974 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.531040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c3d07-ca39-4c0d-95d5-e0885f7a2974 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861/users/bf4d4f6d05b846a9b37142cd6195bb44/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.533061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c3d07-ca39-4c0d-95d5-e0885f7a2974 None admin] RBAC: Authorizing `identity:create_grant(project_id=3267824b193a4a3082969dc9c0836861, user_id=bf4d4f6d05b846a9b37142cd6195bb44, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:19.535247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9c3d07-ca39-4c0d-95d5-e0885f7a2974 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:19.570481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b461753-f6ae-4bd7-a1ed-46dc401d978a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:19.619039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 67/136] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:19 2022] PUT /identity/v3/projects/3267824b193a4a3082969dc9c0836861/users/bf4d4f6d05b846a9b37142cd6195bb44/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:19.620360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056dd1a-ac5a-45fe-9f54-5ef16e257f93 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.624857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056dd1a-ac5a-45fe-9f54-5ef16e257f93 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.625148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056dd1a-ac5a-45fe-9f54-5ef16e257f93 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.628057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 68/137] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:19.631081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4b1ae-640e-4341-b243-3b4ac0a4b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:19.631775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4b1ae-640e-4341-b243-3b4ac0a4b459 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.632056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4b1ae-640e-4341-b243-3b4ac0a4b459 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.632335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4b1ae-640e-4341-b243-3b4ac0a4b459 None admin] PATH_INFO: `/v3/projects/d2e6e396c514416296a551d5165c2d6b/users/a3dfb349996e4ef189c4b08a2a09941e/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.633860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e7964-e10b-46e2-89ca-ee7c221766c5 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.634157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e7964-e10b-46e2-89ca-ee7c221766c5 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.634415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e7964-e10b-46e2-89ca-ee7c221766c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.640158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4b1ae-640e-4341-b243-3b4ac0a4b459 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2e6e396c514416296a551d5165c2d6b, user_id=a3dfb349996e4ef189c4b08a2a09941e, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:19.648880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4b1ae-640e-4341-b243-3b4ac0a4b459 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:19.669583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c14e7964-e10b-46e2-89ca-ee7c221766c5 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:19.670772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c14e7964-e10b-46e2-89ca-ee7c221766c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:19.749115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 70/138] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:19 2022] PUT /identity/v3/projects/d2e6e396c514416296a551d5165c2d6b/users/a3dfb349996e4ef189c4b08a2a09941e/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:54:19.750492 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827ee26-f97e-4302-81d5-3c562ec5da13 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.750781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827ee26-f97e-4302-81d5-3c562ec5da13 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.751060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827ee26-f97e-4302-81d5-3c562ec5da13 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.752750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 71/139] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:19.758520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b8c32-43eb-4c40-b7e5-12a254ef9f5b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.758814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b8c32-43eb-4c40-b7e5-12a254ef9f5b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.761034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b8c32-43eb-4c40-b7e5-12a254ef9f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.774206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 69/140] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:19.788578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8c1b8c32-43eb-4c40-b7e5-12a254ef9f5b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:19.789855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8c1b8c32-43eb-4c40-b7e5-12a254ef9f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:19.837883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e7964-e10b-46e2-89ca-ee7c221766c5 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:19.894275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:19.912484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 72/141] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:19.936093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b8c32-43eb-4c40-b7e5-12a254ef9f5b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:19.999809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.033336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e7964-e10b-46e2-89ca-ee7c221766c5 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.105986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.106658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.106956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.107234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.108900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.110388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.114497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 70/142] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:20.119005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.136011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b8c32-43eb-4c40-b7e5-12a254ef9f5b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.204640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.205223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.205510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.205789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.228498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:20.235206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:20.236045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:20.236806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:20.238435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 73/143] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:20.244160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.319393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8137-4706-4464-93b8-23e060d1fe58 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.353822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.389885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24735805-3957-4cea-a2a3-5a1204939921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.391180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24735805-3957-4cea-a2a3-5a1204939921 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.391462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24735805-3957-4cea-a2a3-5a1204939921 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.391736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24735805-3957-4cea-a2a3-5a1204939921 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.392463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24735805-3957-4cea-a2a3-5a1204939921 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.398267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24735805-3957-4cea-a2a3-5a1204939921 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.442226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c917d-583d-40f8-984d-9a6ffab5a390 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.485693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.486335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.486617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.486895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.487622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] RBAC: Adding query filter params (name=member) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:20.487929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.489968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.498913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 74/144] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:20.504150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.520935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 71/145] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:20 2022] POST /identity/v3/users => generated 283 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:20.545782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5211-2ead-44c8-ad24-86f0a01f51ba None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.546409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5211-2ead-44c8-ad24-86f0a01f51ba None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.546886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5211-2ead-44c8-ad24-86f0a01f51ba None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.548636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 72/146] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:20.563062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba142-5b50-4f5b-9cbb-8d34f140bb60 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.563680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba142-5b50-4f5b-9cbb-8d34f140bb60 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.564147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba142-5b50-4f5b-9cbb-8d34f140bb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.604661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b45ba142-5b50-4f5b-9cbb-8d34f140bb60 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:20.605918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b45ba142-5b50-4f5b-9cbb-8d34f140bb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:20.613817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.687382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f9137-a289-4509-8ef7-3f8ca9e01436 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.700128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.736485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.736485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.736485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.736485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.737228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.738770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.740433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 75/147] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:20.746117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.747130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 73/148] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:20.821504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba142-5b50-4f5b-9cbb-8d34f140bb60 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.875397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29279e4b-927e-4c50-8d89-2dd80021a064 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.914327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.914950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.915229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.915503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.966521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.992855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.992855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:21.093615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 76/149] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:21.099047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:21.146937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba142-5b50-4f5b-9cbb-8d34f140bb60 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:21.181205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:21.202842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4900c9-a193-4561-b53c-0e327dfa80bf None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:21.237906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:21.239115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:21.239398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:21.239672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:21.241036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:21.242722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:21.250924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:21.251080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 77/150] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:21 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:21.251154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:21.251154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:21.251154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:21.251341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:21.270861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:21.285844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:21.290994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:21.291820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:21.292646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 74/151] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:21.321432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:21.375615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec7be4-a3fe-4131-a25f-f9577c0d31a5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:21.413617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:21.414341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:21.414628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:21.414903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:21.415904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=a3dfb349996e4ef189c4b08a2a09941e) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:21.416220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:21.417901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:21.424837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 78/152] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:21 2022] GET /identity/v3/role_assignments?user.id=a3dfb349996e4ef189c4b08a2a09941e&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:21.531715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d917c4-8368-43ec-960a-0b0aa69afe0f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:21.571097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:21.571728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:21.572010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:21.572287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:21.573057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] RBAC: Adding query filter params (name=reader) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:21.573364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:21.574789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:21.585086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 75/153] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:21.590593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:21.608781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:21.701378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b90a5e-7571-4bbd-8bfd-2913500369cd None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:21.737822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:21.738537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:21.738821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:21.739098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] PATH_INFO: `/v3/users/1e2f83a6a3604ec288302cbd0e6c866f` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:21.740406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] RBAC: Authorizing `identity:get_user(user_id=1e2f83a6a3604ec288302cbd0e6c866f)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:21.741881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:21.743686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 79/154] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:21 2022] GET /identity/v3/users/1e2f83a6a3604ec288302cbd0e6c866f => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:21.749040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:21.814276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:21.854995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94735b03-c91e-4a45-ba3c-3bbdad31afde None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:21.890472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300083e-0671-4ed7-94f4-2c9d06f1c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:21.891130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300083e-0671-4ed7-94f4-2c9d06f1c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:21.891413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300083e-0671-4ed7-94f4-2c9d06f1c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:21.891693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300083e-0671-4ed7-94f4-2c9d06f1c3e3 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:21.892904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300083e-0671-4ed7-94f4-2c9d06f1c3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:21.894372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300083e-0671-4ed7-94f4-2c9d06f1c3e3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:21.896659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 76/155] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:21 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:21.902439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:21.985748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:22.007872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc86f2-a5f4-4f2c-8221-c1f768d0d972 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:22.044054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:22.044830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.045251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.045528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861/users/1e2f83a6a3604ec288302cbd0e6c866f/roles/46fcb1e9e111430489c72b70f557e173` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.047526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3267824b193a4a3082969dc9c0836861, user_id=1e2f83a6a3604ec288302cbd0e6c866f, role_id=46fcb1e9e111430489c72b70f557e173)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:22.049051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:22.087115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c31de7-d84b-433c-b5ad-1bc26db580f8 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.087386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c31de7-d84b-433c-b5ad-1bc26db580f8 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.087655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c31de7-d84b-433c-b5ad-1bc26db580f8 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.089209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 77/156] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:22.094547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0860a2-9b40-4536-a1ca-606f9a4439c3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.094890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0860a2-9b40-4536-a1ca-606f9a4439c3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.095301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0860a2-9b40-4536-a1ca-606f9a4439c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.119980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-af0860a2-9b40-4536-a1ca-606f9a4439c3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:22.121211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-af0860a2-9b40-4536-a1ca-606f9a4439c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:22.137843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 80/157] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:21 2022] PUT /identity/v3/projects/3267824b193a4a3082969dc9c0836861/users/1e2f83a6a3604ec288302cbd0e6c866f/roles/46fcb1e9e111430489c72b70f557e173 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:22.207577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 78/158] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:22.230868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:22.396446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab7c9e-e3f7-432b-8686-6271f0cebfe8 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:22.443106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:22.443766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.444046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.444315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.458351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:22.464493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:22.475105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:22.476737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:22.477501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 81/159] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:22.482614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0860a2-9b40-4536-a1ca-606f9a4439c3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:22.557418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:22.600095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0860a2-9b40-4536-a1ca-606f9a4439c3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:22.635720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:22.636330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.636638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.636915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.637649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:22.637959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:22.639361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:22.650557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 79/160] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:22.656065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:22.775071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e283-393e-4b4f-b9ae-f6d50c083fb1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:22.813185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e1073-7bb2-4816-a43b-b8de34ff5c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:22.813837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e1073-7bb2-4816-a43b-b8de34ff5c27 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.814162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e1073-7bb2-4816-a43b-b8de34ff5c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.814446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e1073-7bb2-4816-a43b-b8de34ff5c27 None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.815631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e1073-7bb2-4816-a43b-b8de34ff5c27 None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:22.817635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e1073-7bb2-4816-a43b-b8de34ff5c27 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:22.819244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 82/161] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:22 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:22.824868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:22.944561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ebc51-599f-494a-8dc9-a4810af0ac4a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:22.976409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:22.990462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47980e31-e0b6-4917-8b11-d8ea994f3eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:22.990462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980e31-e0b6-4917-8b11-d8ea994f3eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.990462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980e31-e0b6-4917-8b11-d8ea994f3eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.990462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980e31-e0b6-4917-8b11-d8ea994f3eb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.065357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47980e31-e0b6-4917-8b11-d8ea994f3eb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:23.067598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47980e31-e0b6-4917-8b11-d8ea994f3eb0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:23.150859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9a38-34e0-442f-82be-bff1423bdd98 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.151698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9a38-34e0-442f-82be-bff1423bdd98 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.152147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9a38-34e0-442f-82be-bff1423bdd98 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.153668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 83/162] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:23.169441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80991206-53bd-4fb3-ba58-47807d77c508 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.169979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80991206-53bd-4fb3-ba58-47807d77c508 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.170409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80991206-53bd-4fb3-ba58-47807d77c508 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.171878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 84/163] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:23.173951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b7330-db0b-4158-b4b7-7d022e0610aa None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.174386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b7330-db0b-4158-b4b7-7d022e0610aa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.179179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b7330-db0b-4158-b4b7-7d022e0610aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.207091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 80/164] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:23.208275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b83b7330-db0b-4158-b4b7-7d022e0610aa None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:23.208930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e579a3-46b7-492b-9967-51b53e81e7bd None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.209371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e579a3-46b7-492b-9967-51b53e81e7bd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.209550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b83b7330-db0b-4158-b4b7-7d022e0610aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:23.209964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e579a3-46b7-492b-9967-51b53e81e7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.229441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-41e579a3-46b7-492b-9967-51b53e81e7bd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:23.230925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-41e579a3-46b7-492b-9967-51b53e81e7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:23.253414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:23.319000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 81/165] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:23.320304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e579a3-46b7-492b-9967-51b53e81e7bd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:23.344754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:23.359843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 85/166] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:23.361262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b7330-db0b-4158-b4b7-7d022e0610aa None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:23.485161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e579a3-46b7-492b-9967-51b53e81e7bd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:23.547447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:23.548406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.548869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.549320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.551710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:23.553391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:23.555986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 82/167] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:23 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:23.557033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:23.591485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:23.612315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b7330-db0b-4158-b4b7-7d022e0610aa None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:23.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:23.697772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.698059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.698338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.718416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeef90f-4e27-46cb-be82-d7b344ef86c5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:23.721285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:23.727217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:23.728443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:23.729345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 86/168] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:23.733196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:23.754268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:23.754832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.755107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.755376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.766318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:23.772016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:23.772930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:23.773689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 83/169] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:23.774418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:23.864951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:23.931691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:23.937771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99d2bd-3748-4544-a4c0-be09a273a677 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:23.986603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3283eb-79cb-4603-ad40-f531037342ad None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:24.001671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:24.002310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.002591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.002867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.004424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] RBAC: Adding query filter params (role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=a3dfb349996e4ef189c4b08a2a09941e) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:24.004742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:24.006227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:24.020813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 87/170] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:23 2022] GET /identity/v3/role_assignments?user.id=a3dfb349996e4ef189c4b08a2a09941e&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:24.021766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:24.086722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:24.087374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.087657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.087932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.088728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:24.089037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:24.090522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:24.107877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 84/171] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:24.116404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:24.213368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b954e61-4b07-4d0f-a592-fcfa93b1a952 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:24.256863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:24.280167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:24.281653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.281931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.282208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.282929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:24.283223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:24.284727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:24.304328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 88/172] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:24.310781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:24.374645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b6dd-a8e6-46fe-a8b4-98d686035213 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:24.444738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:24.446502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.446784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.447059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] PATH_INFO: `/v3/users/a3dfb349996e4ef189c4b08a2a09941e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.448913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] RBAC: Authorizing `identity:get_user(user_id=a3dfb349996e4ef189c4b08a2a09941e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:24.450369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:24.453111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 85/173] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:24 2022] GET /identity/v3/users/a3dfb349996e4ef189c4b08a2a09941e => generated 275 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:24.455020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cf593-fb82-4315-bde0-9cf17422422c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:24.457379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:24.481995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:24.495640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:24.496708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.497197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.497609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] PATH_INFO: `/v3/users/bf4d4f6d05b846a9b37142cd6195bb44` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.499079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] RBAC: Authorizing `identity:get_user(user_id=bf4d4f6d05b846a9b37142cd6195bb44)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:24.501236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:24.503192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 89/174] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:24 2022] GET /identity/v3/users/bf4d4f6d05b846a9b37142cd6195bb44 => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:24.509609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:24.596789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:24.674001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7a73-5120-4f3a-858a-5dbef008fb3a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:24.698364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa9ad1-d45f-4362-9705-2c20adce390e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:24.710537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d5765-7f3e-4643-b264-0b33d71f65bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:24.711162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5765-7f3e-4643-b264-0b33d71f65bd None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.711499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5765-7f3e-4643-b264-0b33d71f65bd None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.711779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5765-7f3e-4643-b264-0b33d71f65bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.756643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:24.757385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.757706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.757979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.787379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d5765-7f3e-4643-b264-0b33d71f65bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:24.788862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d5765-7f3e-4643-b264-0b33d71f65bd None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:24.804961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:24.869286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:24.870762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:24.892358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 86/175] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:24.894018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325c663-a564-45db-bd7f-c51385da1dde None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.894404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325c663-a564-45db-bd7f-c51385da1dde None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.894798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325c663-a564-45db-bd7f-c51385da1dde None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.900403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 87/176] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:24.901613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcda195-db6f-4381-ac8d-c0032e31937c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:24.901898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcda195-db6f-4381-ac8d-c0032e31937c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:24.902176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcda195-db6f-4381-ac8d-c0032e31937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:24.921224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9bcda195-db6f-4381-ac8d-c0032e31937c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:24.922393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9bcda195-db6f-4381-ac8d-c0032e31937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:24.984816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:24.991151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 88/177] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:24.996610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcda195-db6f-4381-ac8d-c0032e31937c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.058482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:25.075578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 90/178] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:25.077781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.119459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcda195-db6f-4381-ac8d-c0032e31937c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.155483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.158774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.158774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.158774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] PATH_INFO: `/v3/projects/d2e6e396c514416296a551d5165c2d6b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.158774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] RBAC: Authorizing `identity:get_project(project_id=d2e6e396c514416296a551d5165c2d6b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:25.159330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:25.161786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 89/179] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:24 2022] GET /identity/v3/projects/d2e6e396c514416296a551d5165c2d6b => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:25.162374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.216342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb3968-8c1a-49aa-9d4d-36ba59cb5dd7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.259404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f588a-1e46-4af8-9c1a-b980f97077f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.260647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f588a-1e46-4af8-9c1a-b980f97077f7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.261001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f588a-1e46-4af8-9c1a-b980f97077f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.261283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f588a-1e46-4af8-9c1a-b980f97077f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.263597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440ece9-7d1b-4368-956c-09550cc2d074 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.272747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f588a-1e46-4af8-9c1a-b980f97077f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:25.273355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-ec9f588a-1e46-4af8-9c1a-b980f97077f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:25.279076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:25.286104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 91/180] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:25.286928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d8400-e02e-43bc-8915-894cff9e577a None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.287659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d8400-e02e-43bc-8915-894cff9e577a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.287659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d8400-e02e-43bc-8915-894cff9e577a None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.287659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 92/181] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:25.287659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d8400-e02e-43bc-8915-894cff9e577a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.298721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:25.308010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e220d-da77-4870-baa8-0c5935565cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.308749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e220d-da77-4870-baa8-0c5935565cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.308973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e220d-da77-4870-baa8-0c5935565cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.309243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e220d-da77-4870-baa8-0c5935565cb1 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.310403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e220d-da77-4870-baa8-0c5935565cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:25.311892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e220d-da77-4870-baa8-0c5935565cb1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:25.317324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 90/182] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:25.318823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aa778-df79-40d6-b3d6-f46dfaa53f77 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.319104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aa778-df79-40d6-b3d6-f46dfaa53f77 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.319377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236aa778-df79-40d6-b3d6-f46dfaa53f77 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.320692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 91/183] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:25.324526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec1773-9cb5-44be-974a-3f0be6d7cfaf None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.324526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec1773-9cb5-44be-974a-3f0be6d7cfaf None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.324526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec1773-9cb5-44be-974a-3f0be6d7cfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.347724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1fec1773-9cb5-44be-974a-3f0be6d7cfaf None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:25.348961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1fec1773-9cb5-44be-974a-3f0be6d7cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:25.401778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d8400-e02e-43bc-8915-894cff9e577a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.418330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 92/184] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:25.419476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec1773-9cb5-44be-974a-3f0be6d7cfaf None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.453172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.453948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.454977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.455256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.456045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=d2e6e396c514416296a551d5165c2d6b, user.id=a3dfb349996e4ef189c4b08a2a09941e) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:25.456926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:25.458375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:25.464983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 93/185] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/role_assignments?user.id=a3dfb349996e4ef189c4b08a2a09941e&scope.project.id=d2e6e396c514416296a551d5165c2d6b&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:25.465785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.529681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec1773-9cb5-44be-974a-3f0be6d7cfaf None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.538558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:25.568844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505a6bb-0ae9-4437-949a-055bfc07309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.569413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505a6bb-0ae9-4437-949a-055bfc07309a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.570403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505a6bb-0ae9-4437-949a-055bfc07309a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.570403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505a6bb-0ae9-4437-949a-055bfc07309a None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.570698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505a6bb-0ae9-4437-949a-055bfc07309a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:25.571015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505a6bb-0ae9-4437-949a-055bfc07309a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:25.572518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505a6bb-0ae9-4437-949a-055bfc07309a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:25.582230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:25.588249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 93/186] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:25.588249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18933-44d5-4980-a94c-797e74c814ba None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.588249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18933-44d5-4980-a94c-797e74c814ba None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.588249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c18933-44d5-4980-a94c-797e74c814ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.606141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f8c18933-44d5-4980-a94c-797e74c814ba None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:25.607422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f8c18933-44d5-4980-a94c-797e74c814ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:25.615137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66d6d9-4cdd-41f7-8d22-e2c423c8ef9d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.658621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.659374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.659746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.660090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.661003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=bf4d4f6d05b846a9b37142cd6195bb44) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:25.661369 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:25.662955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:25.670261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 94/187] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/role_assignments?user.id=bf4d4f6d05b846a9b37142cd6195bb44&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:25.671914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.710897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 94/188] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:25.711775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c18933-44d5-4980-a94c-797e74c814ba None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.818417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b7d79-fae5-4fc7-b129-33260743f4ec None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.838373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:25.841321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c18933-44d5-4980-a94c-797e74c814ba None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:25.857753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.858408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.858693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.858964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.860121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:25.861686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:25.863795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 95/189] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:25.864689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.879005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:25.887355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f952f-e447-4f2b-999a-ea1a34452ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:25.887355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f952f-e447-4f2b-999a-ea1a34452ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.887355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f952f-e447-4f2b-999a-ea1a34452ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.889303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f952f-e447-4f2b-999a-ea1a34452ce0 None admin] PATH_INFO: `/v3/users/13735459aae1489cabadc07875065e1c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.902590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f952f-e447-4f2b-999a-ea1a34452ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=13735459aae1489cabadc07875065e1c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:25.904057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f952f-e447-4f2b-999a-ea1a34452ce0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:25.905970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 95/190] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/users/13735459aae1489cabadc07875065e1c => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:25.906968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065f33-4a53-4d22-873b-1a12ccec44f0 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:25.907244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065f33-4a53-4d22-873b-1a12ccec44f0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:25.907516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065f33-4a53-4d22-873b-1a12ccec44f0 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:25.909364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 96/191] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:25.909528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065f33-4a53-4d22-873b-1a12ccec44f0 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:25.977341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d652a7-2b4f-4a80-b945-e756c0075492 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.010557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.011196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.011474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.011572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065f33-4a53-4d22-873b-1a12ccec44f0 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.012023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.021826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:26.027251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:26.028081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:26.028884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 96/192] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:26.029535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.050657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11db7e8-bbde-4684-8f25-1dbe4f4b6517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.051285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11db7e8-bbde-4684-8f25-1dbe4f4b6517 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.051570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11db7e8-bbde-4684-8f25-1dbe4f4b6517 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.051846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11db7e8-bbde-4684-8f25-1dbe4f4b6517 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.052565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11db7e8-bbde-4684-8f25-1dbe4f4b6517 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:26.054273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11db7e8-bbde-4684-8f25-1dbe4f4b6517 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:26.095033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.129507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba1aa0-dc2c-4135-8d72-fc31243b33d3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.163413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 97/193] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:25 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:26.163413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047924-fabe-4df9-b440-3e3f600ef8b9 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.163413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047924-fabe-4df9-b440-3e3f600ef8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.163413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e047924-fabe-4df9-b440-3e3f600ef8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.167423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.168079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.168356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.168671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.188829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-8e047924-fabe-4df9-b440-3e3f600ef8b9 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:26.190135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8e047924-fabe-4df9-b440-3e3f600ef8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:26.200213 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.254824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:26.256982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:26.263293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 98/194] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:26.264336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e047924-fabe-4df9-b440-3e3f600ef8b9 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.338681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.382117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 97/195] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:26.383218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e047924-fabe-4df9-b440-3e3f600ef8b9 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.385224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.433194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.433821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.434097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.434367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.435096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:26.435399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:26.440291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:26.449970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 99/196] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:26.451761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.460863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.512083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d4d87-cd2b-43fb-aa50-7bf600c359f9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.564344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.565750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.566605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.567076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.582930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ce816-ca57-48b9-93d4-24e43f22af92 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.583567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:26.590161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:26.591022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:26.591783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:26.594151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 98/197] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:26.595600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.626765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.627678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.628089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.628547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.629861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:26.631413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:26.632083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.644028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 100/198] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:26.644028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.714444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.737766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726049d-d371-4681-b47c-a97c12aa6803 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.749429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf986a5-0f58-47d7-a884-22c2d9add1d5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.787884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.788543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.788910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.789190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.789623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.789924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:26.790230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:26.790730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.791144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.791550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] PATH_INFO: `/v3/users/13735459aae1489cabadc07875065e1c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.791738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:26.793288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] RBAC: Authorizing `identity:get_user(user_id=13735459aae1489cabadc07875065e1c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:26.795313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:26.797708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 99/199] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/users/13735459aae1489cabadc07875065e1c => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:26.799000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.801066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 101/200] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:26.809472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.917591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae97dc3-bbd8-4073-a94a-29e75df19a52 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.920725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.943471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6eaa4e-12b8-4a21-91a7-62bf23493514 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.958036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.959143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.959430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.959713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.960826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=13735459aae1489cabadc07875065e1c) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:26.961181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:26.963475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:26.976414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 100/201] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/role_assignments?user.id=13735459aae1489cabadc07875065e1c&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:26.976414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.005131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.006174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.007484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.007976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] PATH_INFO: `/v3/users/1e2f83a6a3604ec288302cbd0e6c866f` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.013256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] RBAC: Authorizing `identity:get_user(user_id=1e2f83a6a3604ec288302cbd0e6c866f)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.015569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.017547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 102/202] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/users/1e2f83a6a3604ec288302cbd0e6c866f => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:27.025863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.038773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.101135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725259b-bae1-407b-9b02-a796b268b8ce None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.140984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.142092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.142387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.142668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.272726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.303773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5fb12-f465-412e-8825-45eb577a0b33 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.355184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.356020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.357790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.358077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.358347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.358952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.481767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.483344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.485691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.595505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 103/203] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:27.600630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.620827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 101/204] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:27.627037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.748743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.797942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdc823-50bd-48cc-a8d9-e7242056c3bc None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.804729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.838612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acefe2-f091-42ca-a116-2aa58c142a4a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.859838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.860429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.860717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.860992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.862816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.864415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.866531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 104/205] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:27.872346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.912885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.913535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.913813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.914086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.916360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.917826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.919488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 102/206] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:27.926080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.035588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c0a15-6a92-44b5-ab2c-143c066595e4 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.056615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0f1e7-c6e0-4cd0-ac45-31412bae04d1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.084946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.101436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce2c10-2a8b-41c4-9c7b-576d75c9266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.104650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce2c10-2a8b-41c4-9c7b-576d75c9266c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.104955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce2c10-2a8b-41c4-9c7b-576d75c9266c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.105238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce2c10-2a8b-41c4-9c7b-576d75c9266c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.106054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce2c10-2a8b-41c4-9c7b-576d75c9266c None admin] RBAC: Adding query filter params (role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=13735459aae1489cabadc07875065e1c) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:28.106370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce2c10-2a8b-41c4-9c7b-576d75c9266c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.107840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce2c10-2a8b-41c4-9c7b-576d75c9266c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.109116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9dd67-57ae-4279-a3d2-69261d2bc94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.109736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9dd67-57ae-4279-a3d2-69261d2bc94f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.110065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9dd67-57ae-4279-a3d2-69261d2bc94f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.110286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9dd67-57ae-4279-a3d2-69261d2bc94f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.111089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9dd67-57ae-4279-a3d2-69261d2bc94f None admin] RBAC: Adding query filter params (role.id=46fcb1e9e111430489c72b70f557e173, scope.project.id=3267824b193a4a3082969dc9c0836861, user.id=1e2f83a6a3604ec288302cbd0e6c866f) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:28.111399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9dd67-57ae-4279-a3d2-69261d2bc94f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.112908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9dd67-57ae-4279-a3d2-69261d2bc94f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.128902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 103/207] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/role_assignments?user.id=13735459aae1489cabadc07875065e1c&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.131328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 105/208] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/role_assignments?user.id=1e2f83a6a3604ec288302cbd0e6c866f&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=46fcb1e9e111430489c72b70f557e173 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.431956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e2d3c-d156-41a7-a973-5d0ec3886716 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.432485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e2d3c-d156-41a7-a973-5d0ec3886716 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.432485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e2d3c-d156-41a7-a973-5d0ec3886716 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.433979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 104/209] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:28.439328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3da4a-f412-4258-b878-da0054814d92 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.439858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3da4a-f412-4258-b878-da0054814d92 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.440970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3da4a-f412-4258-b878-da0054814d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.460881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d0a3da4a-f412-4258-b878-da0054814d92 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:28.462374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d0a3da4a-f412-4258-b878-da0054814d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:28.482141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47cf8e-3d4a-4bdb-9ad7-e2df533ec41d None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.482756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47cf8e-3d4a-4bdb-9ad7-e2df533ec41d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.483237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47cf8e-3d4a-4bdb-9ad7-e2df533ec41d None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.484821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 105/210] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:28.493356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e667151-b969-4817-bfaf-50c0bee8fff5 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.493925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e667151-b969-4817-bfaf-50c0bee8fff5 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.494434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e667151-b969-4817-bfaf-50c0bee8fff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.495998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.516167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7e667151-b969-4817-bfaf-50c0bee8fff5 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:28.517802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7e667151-b969-4817-bfaf-50c0bee8fff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:28.559406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 106/211] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:28.594083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3da4a-f412-4258-b878-da0054814d92 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.601173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 106/212] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:28.631848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e667151-b969-4817-bfaf-50c0bee8fff5 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.726585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3da4a-f412-4258-b878-da0054814d92 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.770726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e667151-b969-4817-bfaf-50c0bee8fff5 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.771536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.772596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.773181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.773650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.775029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.776728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.778653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 107/213] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.785760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.799339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.817196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.818144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.818427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.818702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.827947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.832274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:28.838655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:28.839535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:28.840291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:28.843456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 107/214] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:28.850252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.913872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f583e9-fdbb-4e2c-bac0-42ba262dbed8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.962167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.963254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.964025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.964652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.966090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.968103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.968835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fbae9-5bd9-4101-8583-dcc1657c0919 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.011156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.012093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.012621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.013094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.014073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:29.014634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.016196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.031198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 108/215] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:29.032338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.092419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 108/216] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:28 2022] POST /identity/v3/users => generated 297 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:54:29.094587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.165667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a55a-09d4-4e90-9ab0-c8ae8af881f0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.204481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6ea95-7e7c-4caa-b518-2fa6a729bced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.205188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ea95-7e7c-4caa-b518-2fa6a729bced None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.206024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ea95-7e7c-4caa-b518-2fa6a729bced None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.206302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ea95-7e7c-4caa-b518-2fa6a729bced None admin] PATH_INFO: `/v3/users/7ec2fbe52978467caae65aced7e7fd6c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.221638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6ea95-7e7c-4caa-b518-2fa6a729bced None admin] RBAC: Authorizing `identity:get_user(user_id=7ec2fbe52978467caae65aced7e7fd6c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.223249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6ea95-7e7c-4caa-b518-2fa6a729bced None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.225328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 109/217] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/users/7ec2fbe52978467caae65aced7e7fd6c => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:29.231584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.294204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2701-a7a6-4953-ba9c-4b27db1aa64b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.294441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2701-a7a6-4953-ba9c-4b27db1aa64b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.294688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac2701-a7a6-4953-ba9c-4b27db1aa64b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.296045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 110/218] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:29.301142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a5e6c-413a-4657-aa7f-aafa281ee8ec None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.301431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a5e6c-413a-4657-aa7f-aafa281ee8ec None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.301716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a5e6c-413a-4657-aa7f-aafa281ee8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.328900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-dc2a5e6c-413a-4657-aa7f-aafa281ee8ec None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:29.330976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-dc2a5e6c-413a-4657-aa7f-aafa281ee8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:29.338525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.354235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173f588-b097-45f1-8eae-b711fb10a5da None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.392262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.393477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.393838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.394113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.422155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 111/219] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:29.446662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a5e6c-413a-4657-aa7f-aafa281ee8ec None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.483048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.483888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.498297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.583163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a5e6c-413a-4657-aa7f-aafa281ee8ec None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.596596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.611285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 109/220] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:29.617347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.619572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.620248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.620460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.620784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.633337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:29.638649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:29.639513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:29.640281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 112/221] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:29.645791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.732665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f670814-3078-49b9-9eb0-352829279070 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.740516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.750180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38d7de-3e41-4bf9-a174-0757fab19c2a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.771483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.772535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.773012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.773496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.774899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.776588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.778759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 110/222] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:29.785484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.785615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.792413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.792413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.792413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.792413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:29.792413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.792413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.798332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 113/223] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:29.804543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.889815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.907343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b82d5-43c5-4716-9b47-e9272837cff6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.948406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.948406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.948406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.948406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] PATH_INFO: `/v3/users/13735459aae1489cabadc07875065e1c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.948814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] RBAC: Authorizing `identity:get_user(user_id=13735459aae1489cabadc07875065e1c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.948814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.950063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 114/224] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/users/13735459aae1489cabadc07875065e1c => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:29.955492 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.987722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ec997-b91c-43b6-b55d-4fbbcc95aeb8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:30.020829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:30.070495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f86d-e4b1-4c35-ba0b-e20b21822e9f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:30.108597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:30.109394 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:30.109741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:30.110079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:30.112041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:30.114322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:30.115617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 115/225] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:30.121241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:30.124455 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13f36f-fd3d-4e37-8baa-d4ab937315fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:30.130971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13f36f-fd3d-4e37-8baa-d4ab937315fb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:30.130971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13f36f-fd3d-4e37-8baa-d4ab937315fb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:30.130971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13f36f-fd3d-4e37-8baa-d4ab937315fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:30.130971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13f36f-fd3d-4e37-8baa-d4ab937315fb None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=7ec2fbe52978467caae65aced7e7fd6c) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:30.130971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13f36f-fd3d-4e37-8baa-d4ab937315fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:30.130971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13f36f-fd3d-4e37-8baa-d4ab937315fb None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:30.159069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 111/226] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/role_assignments?user.id=7ec2fbe52978467caae65aced7e7fd6c&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 259 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:30.224873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:30.231435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d60001-d47c-4821-a657-a503ccd9f46c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:30.267795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049b897-17a1-4caf-96a6-68e179d97003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:30.268609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b897-17a1-4caf-96a6-68e179d97003 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:30.268894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b897-17a1-4caf-96a6-68e179d97003 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:30.269170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b897-17a1-4caf-96a6-68e179d97003 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/13735459aae1489cabadc07875065e1c/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:30.271389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049b897-17a1-4caf-96a6-68e179d97003 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a18415ea9674b58862ea134ee2ad684, user_id=13735459aae1489cabadc07875065e1c, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:30.272922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049b897-17a1-4caf-96a6-68e179d97003 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:30.387581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 116/227] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:30 2022] PUT /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/13735459aae1489cabadc07875065e1c/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:30.495310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092717b-b7ac-45ad-a86c-662b087e11fe None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:30.495645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092717b-b7ac-45ad-a86c-662b087e11fe None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:30.495864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092717b-b7ac-45ad-a86c-662b087e11fe None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:30.497521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 112/228] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:30.503174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8602983-6039-4bdb-bde7-56816ec6022b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:30.503647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8602983-6039-4bdb-bde7-56816ec6022b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:30.503745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8602983-6039-4bdb-bde7-56816ec6022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:30.523031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c8602983-6039-4bdb-bde7-56816ec6022b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:30.524247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c8602983-6039-4bdb-bde7-56816ec6022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:30.615758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 117/229] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:30.640482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092717b-b7ac-45ad-a86c-662b087e11fe None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:30.752732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092717b-b7ac-45ad-a86c-662b087e11fe None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:30.772843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:30.787893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:30.788608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:30.788891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:30.789167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:30.800481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:30.805974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:30.806860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:30.807616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 113/230] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:30.813381 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8602983-6039-4bdb-bde7-56816ec6022b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:30.902026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:30.926599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8602983-6039-4bdb-bde7-56816ec6022b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:30.979689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:30.980334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:30.980662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:30.980940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:30.981674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:30.981981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:30.983371 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:30.996128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 118/231] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:31.001343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:31.169593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112209a6-cc87-40ed-ae4f-43da36d43f6f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:31.213472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:31.214476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.215090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.215557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] PATH_INFO: `/v3/users/13735459aae1489cabadc07875065e1c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.221781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] RBAC: Authorizing `identity:get_user(user_id=13735459aae1489cabadc07875065e1c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:31.223452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:31.225589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 114/232] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:31 2022] GET /identity/v3/users/13735459aae1489cabadc07875065e1c => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:31.230955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:31.404148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38f4d8-a8b5-4b4e-bf32-dda46822d589 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:31.454237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a83c9-dbb7-4780-843d-0566d3df582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:31.454812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a83c9-dbb7-4780-843d-0566d3df582f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.455650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a83c9-dbb7-4780-843d-0566d3df582f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.455932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a83c9-dbb7-4780-843d-0566d3df582f None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.461106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a83c9-dbb7-4780-843d-0566d3df582f None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:31.462554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a83c9-dbb7-4780-843d-0566d3df582f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:31.464623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 119/233] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:31 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:31.469909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:31.552453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f08cda-7af6-49f2-ad6d-b8a02cd4e1b9 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.552946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f08cda-7af6-49f2-ad6d-b8a02cd4e1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.553044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f08cda-7af6-49f2-ad6d-b8a02cd4e1b9 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.555043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 120/234] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:31.560313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8f596-5a0f-4386-b5cd-9cbc24d3a00a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.560647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8f596-5a0f-4386-b5cd-9cbc24d3a00a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.560943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8f596-5a0f-4386-b5cd-9cbc24d3a00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.572373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:31.601398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-cdb8f596-5a0f-4386-b5cd-9cbc24d3a00a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:31.602658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cdb8f596-5a0f-4386-b5cd-9cbc24d3a00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:31.604833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:31.627781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967dfb76-cb06-43d5-8208-a2ad8259bed9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:31.663800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:31.664903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.665406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.665910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/13735459aae1489cabadc07875065e1c/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.671386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a18415ea9674b58862ea134ee2ad684, user_id=13735459aae1489cabadc07875065e1c, role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:31.674742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:31.718104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 121/235] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:31.742015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8f596-5a0f-4386-b5cd-9cbc24d3a00a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:31.780208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 115/236] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:31 2022] PUT /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/13735459aae1489cabadc07875065e1c/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:31.871622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8f596-5a0f-4386-b5cd-9cbc24d3a00a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:31.927063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690ef4c-ce0b-4d36-b1ff-5bbe90f7d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:31.927981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690ef4c-ce0b-4d36-b1ff-5bbe90f7d897 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.928259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690ef4c-ce0b-4d36-b1ff-5bbe90f7d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.928560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690ef4c-ce0b-4d36-b1ff-5bbe90f7d897 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.942784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:31.949443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690ef4c-ce0b-4d36-b1ff-5bbe90f7d897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-d690ef4c-ce0b-4d36-b1ff-5bbe90f7d897 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:31.954601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:31.959754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:31.960550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 122/237] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:31.965508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:31.982162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294636bd-d188-4554-8e9e-40213b13fabe None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.982484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294636bd-d188-4554-8e9e-40213b13fabe None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.982720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294636bd-d188-4554-8e9e-40213b13fabe None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.985082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 123/238] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:31.990012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4017508-0662-4b2e-9a10-15ebdd8a49dd None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.990297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4017508-0662-4b2e-9a10-15ebdd8a49dd None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.990576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4017508-0662-4b2e-9a10-15ebdd8a49dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.018731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d4017508-0662-4b2e-9a10-15ebdd8a49dd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:32.019922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d4017508-0662-4b2e-9a10-15ebdd8a49dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:32.110549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1ee4c-03b2-4736-8591-f1c980eb7965 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.113114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 124/239] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:32.122772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.153008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.153645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.153927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.154207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.154689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4017508-0662-4b2e-9a10-15ebdd8a49dd None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.154952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:32.155253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.156680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.168176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 116/240] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:32.178152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.301158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea5a1e-5035-4463-a045-ccba55b19800 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.308785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4017508-0662-4b2e-9a10-15ebdd8a49dd None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.360240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.372097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.373127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.373412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.373690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] PATH_INFO: `/v3/users/9e34aff5abf24dab96ab460beb3dc256` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.377238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.377238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.377238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.377238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.379421 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.380962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.383997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 125/241] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:32.389296 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.406495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] RBAC: Authorizing `identity:get_user(user_id=9e34aff5abf24dab96ab460beb3dc256)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.408018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.409930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 117/242] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/users/9e34aff5abf24dab96ab460beb3dc256 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:32.417228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.467967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.550208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea383b07-9f98-4c24-a5f8-fb3fcb71d279 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.588847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.589821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.590299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.590765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.603214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e85b93-3b1b-4cb3-a7cf-183f2a55adb9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.658141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa8e33-5ded-43b3-9e88-54249fc59471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.658764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa8e33-5ded-43b3-9e88-54249fc59471 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.659043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa8e33-5ded-43b3-9e88-54249fc59471 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.659314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa8e33-5ded-43b3-9e88-54249fc59471 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.660083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa8e33-5ded-43b3-9e88-54249fc59471 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.661465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa8e33-5ded-43b3-9e88-54249fc59471 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.667573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.669303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.700100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.775430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 118/243] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:32.781229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.799479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 126/244] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:32 2022] POST /identity/v3/users => generated 293 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:32.805992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385078e1-ff6f-463a-97e7-e993ff4a6928 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.806274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385078e1-ff6f-463a-97e7-e993ff4a6928 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.806559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385078e1-ff6f-463a-97e7-e993ff4a6928 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.808168 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 127/245] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:32.814771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4580f1-89ac-4456-9349-d8ed4124506b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.815052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4580f1-89ac-4456-9349-d8ed4124506b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.815507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4580f1-89ac-4456-9349-d8ed4124506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.835455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-de4580f1-89ac-4456-9349-d8ed4124506b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:32.837007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-de4580f1-89ac-4456-9349-d8ed4124506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:32.890391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.898987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b729cb-a1e8-483e-901b-ac3485de753c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.919455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 128/246] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:32.945370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.955245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.956258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.956877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.957352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.958882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.964915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.967592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 119/247] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:32.973738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4580f1-89ac-4456-9349-d8ed4124506b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.021512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.086324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4580f1-89ac-4456-9349-d8ed4124506b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.124302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7be94-6e4c-4f43-b6ba-0d9cca03a6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.124956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7be94-6e4c-4f43-b6ba-0d9cca03a6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.125245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7be94-6e4c-4f43-b6ba-0d9cca03a6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.125507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7be94-6e4c-4f43-b6ba-0d9cca03a6a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.126839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7be94-6e4c-4f43-b6ba-0d9cca03a6a8 None admin] RBAC: Adding query filter params (role.id=46fcb1e9e111430489c72b70f557e173, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=9e34aff5abf24dab96ab460beb3dc256) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:33.127115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7be94-6e4c-4f43-b6ba-0d9cca03a6a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.129567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7be94-6e4c-4f43-b6ba-0d9cca03a6a8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.135710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 129/248] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/role_assignments?user.id=9e34aff5abf24dab96ab460beb3dc256&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=46fcb1e9e111430489c72b70f557e173 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.150063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a96bed-2cf3-4f0f-909b-0e1db63eff77 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.150351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a96bed-2cf3-4f0f-909b-0e1db63eff77 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.150542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a96bed-2cf3-4f0f-909b-0e1db63eff77 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.152596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 130/249] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:33.157650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f1a27-8da8-49d5-960f-7c7b94b563e4 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.159213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705327-7ac4-43eb-9ad0-b3cd4ea538b8 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.159596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705327-7ac4-43eb-9ad0-b3cd4ea538b8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.160125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705327-7ac4-43eb-9ad0-b3cd4ea538b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.186155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6d705327-7ac4-43eb-9ad0-b3cd4ea538b8 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:33.188323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6d705327-7ac4-43eb-9ad0-b3cd4ea538b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:33.209423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.210694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.215770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.215770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.225257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:33.231503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:33.232703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:33.233671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:33.235792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 120/250] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:33.244219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.252805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.291308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 131/251] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:33.315150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d705327-7ac4-43eb-9ad0-b3cd4ea538b8 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.344785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.396104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a39ab-a758-414c-aabd-732b469ff445 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.443760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.444322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.444659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.444946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.445685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:33.445999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.447500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.457855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 121/252] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.458942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d705327-7ac4-43eb-9ad0-b3cd4ea538b8 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.463302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.501109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.502700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.503139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.503557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.518415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:33.524924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:33.525779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:33.526611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 132/253] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:33.533564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.584829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.642778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470705a-3bef-41b5-8314-b5df06f4e306 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.668155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ab928-02c2-49b9-98e5-4ac80ab8a755 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.678493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.679118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.679404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.679678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.680424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:33.680734 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.682127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.691679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 133/254] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.701399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.720876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.733706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.744469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.744469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.744469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] PATH_INFO: `/v3/users/7ec2fbe52978467caae65aced7e7fd6c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.744469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] RBAC: Authorizing `identity:get_user(user_id=7ec2fbe52978467caae65aced7e7fd6c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.744469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.744469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 122/255] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/users/7ec2fbe52978467caae65aced7e7fd6c => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:33.746195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.876930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.883565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcf5fc-0c4f-4395-a6e8-4b90031b32ff None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.951973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.952567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.952852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.953126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] PATH_INFO: `/v3/users/13735459aae1489cabadc07875065e1c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.955123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] RBAC: Authorizing `identity:get_user(user_id=13735459aae1489cabadc07875065e1c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.956608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.958253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 134/256] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/users/13735459aae1489cabadc07875065e1c => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.964600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.007088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d33369-15b5-402b-9857-c7d6b9d98c67 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.080119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.080795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.081075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.081349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.083458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.084942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.086612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 123/257] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:34.093036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.161415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c557fd9-637f-499e-8d3f-983edf409300 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.198005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.198635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.198917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.199195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.223798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.276230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.309465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9d1a-a6f9-4809-95ae-1184df6c78c9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.319167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.320679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.385074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d8dd3-bd8c-4a79-a55c-85b12b915166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.385759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d8dd3-bd8c-4a79-a55c-85b12b915166 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.386042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d8dd3-bd8c-4a79-a55c-85b12b915166 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.386322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d8dd3-bd8c-4a79-a55c-85b12b915166 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/7ec2fbe52978467caae65aced7e7fd6c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.389599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d8dd3-bd8c-4a79-a55c-85b12b915166 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a18415ea9674b58862ea134ee2ad684, user_id=7ec2fbe52978467caae65aced7e7fd6c, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.391051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d8dd3-bd8c-4a79-a55c-85b12b915166 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.478068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 124/258] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:34 2022] PUT /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/7ec2fbe52978467caae65aced7e7fd6c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:34.492077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cbdb2-00ca-4e3b-88eb-cc2ddb433f20 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.492437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cbdb2-00ca-4e3b-88eb-cc2ddb433f20 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.492769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cbdb2-00ca-4e3b-88eb-cc2ddb433f20 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.494398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 125/259] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:34.498241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 135/260] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:34.500616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51c536-8d93-48c7-8090-a75c75dd4107 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.500957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51c536-8d93-48c7-8090-a75c75dd4107 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.501279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51c536-8d93-48c7-8090-a75c75dd4107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.507329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.531855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-af51c536-8d93-48c7-8090-a75c75dd4107 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:34.533887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-af51c536-8d93-48c7-8090-a75c75dd4107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:34.646481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.672452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 126/261] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:34.695048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227b213-3925-4760-a2a8-75bd236db1c1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.727898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51c536-8d93-48c7-8090-a75c75dd4107 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.742423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.743389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.744054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.745991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.748160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.749861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.752635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 136/262] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:34 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:34.767701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.832849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.948602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51c536-8d93-48c7-8090-a75c75dd4107 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.022338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.022917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.023195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.023467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.033531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ef68d-d8df-45c4-a412-307f1bed001d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.047662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:35.052989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:35.053832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:35.054606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:35.055189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 127/263] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:35.061793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:35.169675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.170324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.170614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.170887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.171954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=13735459aae1489cabadc07875065e1c) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:35.171954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.174737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.176984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:35.194395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 137/264] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:34 2022] GET /identity/v3/role_assignments?user.id=13735459aae1489cabadc07875065e1c&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:35.195131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a701c2-d969-4831-a485-43162ad47375 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.216408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:35.252634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.253972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.254398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.254827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.255693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:35.256148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.257718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.268112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 128/265] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:35.273953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:35.478162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf36aa-2c82-4ca4-9a23-e9d4609e415d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.542113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.542711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.542996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.543273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] PATH_INFO: `/v3/users/13735459aae1489cabadc07875065e1c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.544832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] RBAC: Authorizing `identity:get_user(user_id=13735459aae1489cabadc07875065e1c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.545964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.547576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 138/266] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:35 2022] GET /identity/v3/users/13735459aae1489cabadc07875065e1c => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:35.560165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:35.681878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc6983-7440-4461-8331-0efda07bacbf None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.717103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f308e3e-e17f-461e-8e40-8ae50d7e2788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.718131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308e3e-e17f-461e-8e40-8ae50d7e2788 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.718605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308e3e-e17f-461e-8e40-8ae50d7e2788 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.719068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308e3e-e17f-461e-8e40-8ae50d7e2788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.757245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:35.804281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f308e3e-e17f-461e-8e40-8ae50d7e2788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.805760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f308e3e-e17f-461e-8e40-8ae50d7e2788 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.941564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 129/267] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:35.947508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.060859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:36.095962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33b752-db28-45a6-8eee-5b89e3594384 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:36.113168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5efe8-ed54-4669-87a3-1a285bbf7d49 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.113797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5efe8-ed54-4669-87a3-1a285bbf7d49 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.114276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5efe8-ed54-4669-87a3-1a285bbf7d49 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.119523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 130/268] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:36.124254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898fd3-98e1-4004-9caa-b3551510d3ad None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.124781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898fd3-98e1-4004-9caa-b3551510d3ad None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.125275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df898fd3-98e1-4004-9caa-b3551510d3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.130863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:36.131506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.131791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.132066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.133266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:36.134886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:36.137014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 139/269] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:35 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:36.143105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.151918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-df898fd3-98e1-4004-9caa-b3551510d3ad None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:36.153454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-df898fd3-98e1-4004-9caa-b3551510d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:36.236696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:36.245938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 131/270] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:36.248146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7b22f-5a50-4c71-a812-f1699007395b None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.248647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7b22f-5a50-4c71-a812-f1699007395b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.249632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7b22f-5a50-4c71-a812-f1699007395b None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.251773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 132/271] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:36.256954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30951e-d940-477f-b145-d644d5231582 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.257497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30951e-d940-477f-b145-d644d5231582 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.257497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30951e-d940-477f-b145-d644d5231582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.276199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089568a5-6018-4faa-be52-7b01921b2f90 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:36.280408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fb30951e-d940-477f-b145-d644d5231582 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:36.281704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fb30951e-d940-477f-b145-d644d5231582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:36.313954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:36.314844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.315320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.315767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.316819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] RBAC: Adding query filter params (role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=13735459aae1489cabadc07875065e1c) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:36.317301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:36.319019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:36.321119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:36.325756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 140/272] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:36 2022] GET /identity/v3/role_assignments?user.id=13735459aae1489cabadc07875065e1c&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:36.326634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.384163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 133/273] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:36.387070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7f770-62b6-41e3-9aea-36c4f0e6119e None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.387692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7f770-62b6-41e3-9aea-36c4f0e6119e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.388160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7f770-62b6-41e3-9aea-36c4f0e6119e None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.391908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 134/274] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:36.399722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7a53-dd73-4ba8-b49f-a3ff3538a393 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.400029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7a53-dd73-4ba8-b49f-a3ff3538a393 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.400266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7a53-dd73-4ba8-b49f-a3ff3538a393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.430351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f7ae7a53-dd73-4ba8-b49f-a3ff3538a393 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:36.432306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f7ae7a53-dd73-4ba8-b49f-a3ff3538a393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:36.481962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f270667-c88b-44c4-8a09-61d2fb6e254e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:36.525859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:36.526888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.527165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.527435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.534169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 135/275] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:36.535441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7a53-dd73-4ba8-b49f-a3ff3538a393 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.545687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:36.551584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:36.552479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:36.553266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 141/276] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:36.562876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.617706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:36.636329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:36.660237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7a53-dd73-4ba8-b49f-a3ff3538a393 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:36.690327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509db6d-22aa-4bfa-abf7-8b91719c661d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:36.694413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:36.695045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.695325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.695598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.706838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:36.711852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:36.713007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:36.713776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:36.714369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 136/277] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:36.715586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.733268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:36.733901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.734177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.734443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.735495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:36.735803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:36.737195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:36.745966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 142/278] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:36.751496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.828564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d14984-c781-494d-a7c8-0db935dd1905 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:36.845082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:36.858661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3c09f-e646-41a8-8744-fd4dc1708e82 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:36.861871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:36.862511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.862789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.863064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.864350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:36.866157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:36.868168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 137/279] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:36.868915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.884058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:36.899249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:36.900152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:36.900674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:36.901149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:36.902176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] RBAC: Adding query filter params (name=member) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:36.902657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:36.904277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:36.913850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 143/280] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:36.914815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:36.978708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab725fa-a7c6-437b-9e80-abf283b11606 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.015153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.015820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.016111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.017040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] PATH_INFO: `/v3/users/9e34aff5abf24dab96ab460beb3dc256` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.018283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] RBAC: Authorizing `identity:get_user(user_id=9e34aff5abf24dab96ab460beb3dc256)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.019730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.021364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 138/281] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:36 2022] GET /identity/v3/users/9e34aff5abf24dab96ab460beb3dc256 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:37.022125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.029270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e6bdd-1918-47fe-9278-0987ae2b5aea None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.065659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.066143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.066420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.066693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.067410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.068850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.078012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.126341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061c2e9-307e-4155-bba4-6a3eaa572b1f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.159058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.159694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.159973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.160263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] PATH_INFO: `/v3/users/e1e788b101514adb8db4fd793b954135` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.174270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] RBAC: Authorizing `identity:get_user(user_id=e1e788b101514adb8db4fd793b954135)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.175757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.182317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 139/282] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/users/e1e788b101514adb8db4fd793b954135 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:37.182317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.202030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 144/283] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:36 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:37.203088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.261146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.326365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56e3e7-03c6-4a7e-b9e5-2cb490c11597 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.341308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.342514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b1aff-cf7b-4fd0-ba16-28edada59025 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.379473 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41480a04-e509-493b-a720-2f1a1a50d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.380690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41480a04-e509-493b-a720-2f1a1a50d808 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.381188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41480a04-e509-493b-a720-2f1a1a50d808 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.382074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41480a04-e509-493b-a720-2f1a1a50d808 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.383845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41480a04-e509-493b-a720-2f1a1a50d808 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.385749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41480a04-e509-493b-a720-2f1a1a50d808 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.386870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3f491-17a6-4be6-9baf-3a8f6a8375a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.387498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3f491-17a6-4be6-9baf-3a8f6a8375a2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.387786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3f491-17a6-4be6-9baf-3a8f6a8375a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.387969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 140/284] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:37.388121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3f491-17a6-4be6-9baf-3a8f6a8375a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.388969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3f491-17a6-4be6-9baf-3a8f6a8375a2 None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, user.id=e1e788b101514adb8db4fd793b954135, scope.system=all) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:37.389089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18119b7b-50b7-45bc-b0c6-fcd634af56be None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.389284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3f491-17a6-4be6-9baf-3a8f6a8375a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.389377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18119b7b-50b7-45bc-b0c6-fcd634af56be None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.389634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18119b7b-50b7-45bc-b0c6-fcd634af56be None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.390735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3f491-17a6-4be6-9baf-3a8f6a8375a2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.390835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 141/285] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:37.394117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18119b7b-50b7-45bc-b0c6-fcd634af56be None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.396795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 145/286] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/role_assignments?user.id=e1e788b101514adb8db4fd793b954135&scope.system=all&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:37.398731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9d3d1-085d-4f13-8059-24e14c407894 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.398731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9d3d1-085d-4f13-8059-24e14c407894 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.398923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9d3d1-085d-4f13-8059-24e14c407894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.418773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d0e9d3d1-085d-4f13-8059-24e14c407894 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:37.420023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d0e9d3d1-085d-4f13-8059-24e14c407894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:37.523664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 146/287] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:37.538982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18119b7b-50b7-45bc-b0c6-fcd634af56be None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.551867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9d3d1-085d-4f13-8059-24e14c407894 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.585541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.587025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.587025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.587025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/9e34aff5abf24dab96ab460beb3dc256/roles/46fcb1e9e111430489c72b70f557e173` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.589789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a18415ea9674b58862ea134ee2ad684, user_id=9e34aff5abf24dab96ab460beb3dc256, role_id=46fcb1e9e111430489c72b70f557e173)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.591243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.600147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.674842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9d3d1-085d-4f13-8059-24e14c407894 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.682408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 142/288] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:37 2022] PUT /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684/users/9e34aff5abf24dab96ab460beb3dc256/roles/46fcb1e9e111430489c72b70f557e173 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:37.720953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.721617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.721900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.722174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.734314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:37.742974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:37.743908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:37.744747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 147/289] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:37.753300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.799696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.877177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c871b-ca8c-440b-ae70-7240414ae8c2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.928588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.929143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.929429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.929707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.930450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:37.930760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.932171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.941740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 143/290] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:37.957892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.064106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.073767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883123c0-2dc4-4301-827d-5f96ab44adb7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.115188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.116173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.116646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.117140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] PATH_INFO: `/v3/users/7ec2fbe52978467caae65aced7e7fd6c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.118534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] RBAC: Authorizing `identity:get_user(user_id=7ec2fbe52978467caae65aced7e7fd6c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.120125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.121912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 148/291] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/users/7ec2fbe52978467caae65aced7e7fd6c => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:38.127764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.235309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.238406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290b913-0631-48d6-b88a-38c196dd445a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.287876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.288534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.288820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.289143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.367311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.368807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.481775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 144/292] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:38.482363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.487196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.611564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fb78a-b8bc-4383-b43b-02286cfac159 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.655529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268fb18-40b3-4ef9-9e4d-f35260ba1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268fb18-40b3-4ef9-9e4d-f35260ba1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.656578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268fb18-40b3-4ef9-9e4d-f35260ba1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.656855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268fb18-40b3-4ef9-9e4d-f35260ba1bb9 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.658132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268fb18-40b3-4ef9-9e4d-f35260ba1bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.659612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268fb18-40b3-4ef9-9e4d-f35260ba1bb9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.661801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 149/293] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:38.669543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.825161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4abe0c-a94b-4e5c-9198-30442550d1fd None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.878729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc50258-df6c-4419-8fd6-5ed5758f2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.879378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50258-df6c-4419-8fd6-5ed5758f2eaa None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.879661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50258-df6c-4419-8fd6-5ed5758f2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.879940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50258-df6c-4419-8fd6-5ed5758f2eaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.880802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc50258-df6c-4419-8fd6-5ed5758f2eaa None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=7ec2fbe52978467caae65aced7e7fd6c) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:38.881120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc50258-df6c-4419-8fd6-5ed5758f2eaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.882608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc50258-df6c-4419-8fd6-5ed5758f2eaa None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.893296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 145/294] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/role_assignments?user.id=7ec2fbe52978467caae65aced7e7fd6c&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:39.674095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2108eb-07d0-4f54-a3c2-5e7fc6f2e14f None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.674592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2108eb-07d0-4f54-a3c2-5e7fc6f2e14f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.674592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2108eb-07d0-4f54-a3c2-5e7fc6f2e14f None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.676445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 150/295] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:39.681809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31c02-0ba0-4167-a098-55128cb28e15 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.682201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31c02-0ba0-4167-a098-55128cb28e15 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.682386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a31c02-0ba0-4167-a098-55128cb28e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.705313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-35a31c02-0ba0-4167-a098-55128cb28e15 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:39.706496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-35a31c02-0ba0-4167-a098-55128cb28e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:39.767686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.774572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3b235-7b77-4130-b581-5c428823b903 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.774857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3b235-7b77-4130-b581-5c428823b903 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.775167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3b235-7b77-4130-b581-5c428823b903 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.776552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 151/296] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:39.778969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 146/297] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:39.781963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b007-4fcf-460a-a077-fac4fa7aebb0 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.782253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b007-4fcf-460a-a077-fac4fa7aebb0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.782530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b007-4fcf-460a-a077-fac4fa7aebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.803255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-aae4b007-4fcf-460a-a077-fac4fa7aebb0 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:39.803432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a31c02-0ba0-4167-a098-55128cb28e15 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.804594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-aae4b007-4fcf-460a-a077-fac4fa7aebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:39.836314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.885737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 152/298] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:39.886803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed326-c2b6-4b46-bdf1-2a4efd53e011 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.887091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed326-c2b6-4b46-bdf1-2a4efd53e011 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.887366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed326-c2b6-4b46-bdf1-2a4efd53e011 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.888781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 153/299] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:39.894424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820b252-4b32-4b86-b6ee-42d1b2508040 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.894704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820b252-4b32-4b86-b6ee-42d1b2508040 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.894976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820b252-4b32-4b86-b6ee-42d1b2508040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.917077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4820b252-4b32-4b86-b6ee-42d1b2508040 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:39.918256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4820b252-4b32-4b86-b6ee-42d1b2508040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:39.922648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a31c02-0ba0-4167-a098-55128cb28e15 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.956211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.956806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.957088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.957362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.967762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:39.973318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:39.974188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:39.975003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 147/300] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:39.975639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.996481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 154/301] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:39.997592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820b252-4b32-4b86-b6ee-42d1b2508040 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.037250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.085678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cfb00-4567-445d-82bb-89ead4fb252d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.095935 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.107449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820b252-4b32-4b86-b6ee-42d1b2508040 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.121219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.121863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.122144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.122421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.134077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:40.139445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:40.141030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:40.141832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 148/302] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:40.142468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.144993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.145666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.146233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.146510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.147232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:40.147538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.148956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.157909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 155/303] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.163205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.252185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57cfcc-b48e-4510-8169-186ab7712b4a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.267045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.270351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f1f27-b705-447e-a74a-749e06896de5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.288485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.289121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.289402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.289676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.290832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.292260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.293895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 149/304] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.294652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.309799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.310419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.310700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.310974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.311691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:40.311994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.314043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.325706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 156/305] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.326597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.344974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.407627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24c11c-7030-42ac-a82f-16e4ef1ee286 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.427591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76dda-bde1-41ed-90ac-27edb7b437c0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.441908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a1aa1-64ca-46b1-9bef-c106956fa479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.442538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1aa1-64ca-46b1-9bef-c106956fa479 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.442815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1aa1-64ca-46b1-9bef-c106956fa479 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.443087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1aa1-64ca-46b1-9bef-c106956fa479 None admin] PATH_INFO: `/v3/users/701e9561643f4a20a5206fae52e3fc3d` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.456776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a1aa1-64ca-46b1-9bef-c106956fa479 None admin] RBAC: Authorizing `identity:get_user(user_id=701e9561643f4a20a5206fae52e3fc3d)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.458239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a1aa1-64ca-46b1-9bef-c106956fa479 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.459872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 150/306] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/users/701e9561643f4a20a5206fae52e3fc3d => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.464991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.464991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.465169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f5bb-f15e-4f19-8747-578f694b19c2 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.465242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.465308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f5bb-f15e-4f19-8747-578f694b19c2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.465372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] PATH_INFO: `/v3/groups` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.465437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7f5bb-f15e-4f19-8747-578f694b19c2 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.465510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.465577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 151/307] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:40.465577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7f5bb-f15e-4f19-8747-578f694b19c2 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.465698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.506197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.531884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 157/308] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Jul 27 19:54:40 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:40.532732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.569509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7f5bb-f15e-4f19-8747-578f694b19c2 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.609218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98a4c-0b5f-4731-89ee-38fc380e0b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.609875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98a4c-0b5f-4731-89ee-38fc380e0b67 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.610161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98a4c-0b5f-4731-89ee-38fc380e0b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.610433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98a4c-0b5f-4731-89ee-38fc380e0b67 None admin] PATH_INFO: `/v3/users/e1e788b101514adb8db4fd793b954135` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.611746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98a4c-0b5f-4731-89ee-38fc380e0b67 None admin] RBAC: Authorizing `identity:get_user(user_id=e1e788b101514adb8db4fd793b954135)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.613251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98a4c-0b5f-4731-89ee-38fc380e0b67 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.615177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 152/309] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/users/e1e788b101514adb8db4fd793b954135 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.615702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.616247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05df94-d5f4-483c-9963-57edbd08085b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.616790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05df94-d5f4-483c-9963-57edbd08085b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.616790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05df94-d5f4-483c-9963-57edbd08085b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.636435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-5b05df94-d5f4-483c-9963-57edbd08085b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:40.637649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-5b05df94-d5f4-483c-9963-57edbd08085b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:40.655204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657409ad-de2e-4a3e-bfce-d967b37c3a60 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.693583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.694227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.694482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.694760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.695543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] RBAC: Adding query filter params (role.id=46fcb1e9e111430489c72b70f557e173, user.id=701e9561643f4a20a5206fae52e3fc3d, scope.system=all) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:40.695859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.697345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.702548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 158/310] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/role_assignments?user.id=701e9561643f4a20a5206fae52e3fc3d&scope.system=all&role.id=46fcb1e9e111430489c72b70f557e173 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.703978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.715076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 153/311] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:40.737677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05df94-d5f4-483c-9963-57edbd08085b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.798109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.832642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07f484-af9f-49be-980f-56f2b18517bf None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.877170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05df94-d5f4-483c-9963-57edbd08085b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.889962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.891799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.892283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.893048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] PATH_INFO: `/v3/system/users/e1e788b101514adb8db4fd793b954135/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.895475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e1e788b101514adb8db4fd793b954135, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.897799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.903044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.915646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.920170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.920170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.920170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.927950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:40.933090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:40.933936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:40.934729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 154/312] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:40.941311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.996499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 159/313] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Jul 27 19:54:40 2022] PUT /identity/v3/system/users/e1e788b101514adb8db4fd793b954135/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:41.056875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.061729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5d25f-aae2-4e6e-b65f-3880c74e26bb None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.103452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.104089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.104366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.104663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.105402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:41.105698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.107096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.117971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 155/314] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.123592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.244186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf0ca2-f75d-44c6-a886-6d22a5d39ab5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.279973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.280799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.281221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.281499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] PATH_INFO: `/v3/users/9e34aff5abf24dab96ab460beb3dc256` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.282692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] RBAC: Authorizing `identity:get_user(user_id=9e34aff5abf24dab96ab460beb3dc256)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.284140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.285945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 160/315] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/users/9e34aff5abf24dab96ab460beb3dc256 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.291286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.397781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c395ae-8892-43d0-9410-620328a4c754 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.432492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.433399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.433877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.434316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.465225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.503527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.505303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.603963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 156/316] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:41.609791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.680975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.713491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c5b9d-3c73-46dd-92f3-9305fe030537 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.751138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d777bb-3356-4056-9ba0-9b41aa96e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.751854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d777bb-3356-4056-9ba0-9b41aa96e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.752142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d777bb-3356-4056-9ba0-9b41aa96e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.752444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d777bb-3356-4056-9ba0-9b41aa96e4e5 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.753675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d777bb-3356-4056-9ba0-9b41aa96e4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.755134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d777bb-3356-4056-9ba0-9b41aa96e4e5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.757143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 161/317] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.762945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.822476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.866053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8f153-bf46-423e-8f52-17fba11e6b48 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.904241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f99fd2-6a7f-460b-91ae-67ce322a31be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.905148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99fd2-6a7f-460b-91ae-67ce322a31be None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.905647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99fd2-6a7f-460b-91ae-67ce322a31be None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.906090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f99fd2-6a7f-460b-91ae-67ce322a31be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.907048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f99fd2-6a7f-460b-91ae-67ce322a31be None admin] RBAC: Adding query filter params (role.id=46fcb1e9e111430489c72b70f557e173, scope.project.id=2a18415ea9674b58862ea134ee2ad684, user.id=9e34aff5abf24dab96ab460beb3dc256) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:41.907533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f99fd2-6a7f-460b-91ae-67ce322a31be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.909140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f99fd2-6a7f-460b-91ae-67ce322a31be None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.915541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 157/318] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/role_assignments?user.id=9e34aff5abf24dab96ab460beb3dc256&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=46fcb1e9e111430489c72b70f557e173 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:42.677708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205804b-2870-46f5-b6cf-6c0aa951a010 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.678357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205804b-2870-46f5-b6cf-6c0aa951a010 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.678873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205804b-2870-46f5-b6cf-6c0aa951a010 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.680717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 162/319] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:42.685399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a767f0-995d-43a3-bb3c-dd3f930fd522 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.685686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a767f0-995d-43a3-bb3c-dd3f930fd522 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.685957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a767f0-995d-43a3-bb3c-dd3f930fd522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.704783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a2a767f0-995d-43a3-bb3c-dd3f930fd522 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:42.705956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a2a767f0-995d-43a3-bb3c-dd3f930fd522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:42.748843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3e4a6-0875-4e72-88da-cb95aed45bee None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.749908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3e4a6-0875-4e72-88da-cb95aed45bee None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.749985 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3e4a6-0875-4e72-88da-cb95aed45bee None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.751321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 163/320] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:42.756075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7946c3-e59b-4b56-b08e-172534c64ab4 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.756364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7946c3-e59b-4b56-b08e-172534c64ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.756666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7946c3-e59b-4b56-b08e-172534c64ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.775021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 158/321] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:42.775495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-dc7946c3-e59b-4b56-b08e-172534c64ab4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:42.776116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-dc7946c3-e59b-4b56-b08e-172534c64ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:42.796512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a767f0-995d-43a3-bb3c-dd3f930fd522 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.843167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 164/322] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:42.854406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.863949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7946c3-e59b-4b56-b08e-172534c64ab4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.900550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a767f0-995d-43a3-bb3c-dd3f930fd522 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.930476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.938143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.938787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.939069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.939343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.940522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.941999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.943721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 159/323] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:42.949005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.969112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7946c3-e59b-4b56-b08e-172534c64ab4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.004612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.005624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.006158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.006630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.018553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:43.024264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:43.025318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:43.026265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:43.028677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 165/324] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:43.033907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.055907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3c48c-b32f-4f9b-b95d-4c9a78706fb5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.078973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.090335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6b55a-3c36-4f12-9239-3fde57d85d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.091010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b55a-3c36-4f12-9239-3fde57d85d9e None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.091292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b55a-3c36-4f12-9239-3fde57d85d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.091572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b55a-3c36-4f12-9239-3fde57d85d9e None admin] PATH_INFO: `/v3/groups` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.092235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6b55a-3c36-4f12-9239-3fde57d85d9e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.093751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6b55a-3c36-4f12-9239-3fde57d85d9e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.133866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36be165-c14f-4a0a-9968-05719f42268b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.158507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.166615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fd62-d43a-4c84-b19d-0fa473f572f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.167229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fd62-d43a-4c84-b19d-0fa473f572f6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.167509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fd62-d43a-4c84-b19d-0fa473f572f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.167781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fd62-d43a-4c84-b19d-0fa473f572f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.168518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fd62-d43a-4c84-b19d-0fa473f572f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:43.168824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fd62-d43a-4c84-b19d-0fa473f572f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.170239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fd62-d43a-4c84-b19d-0fa473f572f6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.178848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 166/325] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:43.179978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f63d7-f889-4743-a983-f77568370733 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.180259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f63d7-f889-4743-a983-f77568370733 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.180614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f63d7-f889-4743-a983-f77568370733 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.181755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 167/326] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:43.184427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f63d7-f889-4743-a983-f77568370733 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.194752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 160/327] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Jul 27 19:54:42 2022] POST /identity/v3/groups => generated 226 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:43.196140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40c2f1-3bec-4a62-998c-cc2db1bec726 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.196404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40c2f1-3bec-4a62-998c-cc2db1bec726 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.196676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40c2f1-3bec-4a62-998c-cc2db1bec726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.222636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4d40c2f1-3bec-4a62-998c-cc2db1bec726 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:43.223903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4d40c2f1-3bec-4a62-998c-cc2db1bec726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:43.301409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f63d7-f889-4743-a983-f77568370733 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.312347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 161/328] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:43.326572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40c2f1-3bec-4a62-998c-cc2db1bec726 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.341878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.342953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.343245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.343528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] PATH_INFO: `/v3/users/701e9561643f4a20a5206fae52e3fc3d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.345008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] RBAC: Authorizing `identity:get_user(user_id=701e9561643f4a20a5206fae52e3fc3d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.346458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.349111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 168/329] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/users/701e9561643f4a20a5206fae52e3fc3d => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:43.354447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.420469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.454078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40c2f1-3bec-4a62-998c-cc2db1bec726 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.470545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.482432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b7bd-6818-47f0-81ba-d340439740ab None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.491294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.491941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.492222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.492582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.505045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:43.510515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:43.511386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:43.512201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:43.513892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 162/330] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:43.520009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.527451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.528131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.528438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.528724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] PATH_INFO: `/v3/system/users/701e9561643f4a20a5206fae52e3fc3d/roles/46fcb1e9e111430489c72b70f557e173` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.530798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=701e9561643f4a20a5206fae52e3fc3d, role_id=46fcb1e9e111430489c72b70f557e173)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.532196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.602015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 169/331] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Jul 27 19:54:43 2022] PUT /identity/v3/system/users/701e9561643f4a20a5206fae52e3fc3d/roles/46fcb1e9e111430489c72b70f557e173 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:54:43.645745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8ced6-7d10-4bc8-8147-922f94ac9732 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.687780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.688497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.688781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.689054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.689795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:43.690097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.691589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.701484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 163/332] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:43.707399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.823589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1f67-3539-4b32-ae0c-90bc4eb472ad None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.845534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.860725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a89ae8-bcec-4bcf-840e-5e76588e09b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.861387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89ae8-bcec-4bcf-840e-5e76588e09b6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.861671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89ae8-bcec-4bcf-840e-5e76588e09b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.861973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a89ae8-bcec-4bcf-840e-5e76588e09b6 None admin] PATH_INFO: `/v3/users/e1e788b101514adb8db4fd793b954135` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.863140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a89ae8-bcec-4bcf-840e-5e76588e09b6 None admin] RBAC: Authorizing `identity:get_user(user_id=e1e788b101514adb8db4fd793b954135)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.864631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a89ae8-bcec-4bcf-840e-5e76588e09b6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.866608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 170/333] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/users/e1e788b101514adb8db4fd793b954135 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:43.872538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.920904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.998763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c14c0d-b0ce-4761-9ec3-a7260c805b9a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:44.039676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fe0bc-c981-4997-9fac-0448e60fe43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:44.040306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fe0bc-c981-4997-9fac-0448e60fe43d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:44.040649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fe0bc-c981-4997-9fac-0448e60fe43d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:44.040924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fe0bc-c981-4997-9fac-0448e60fe43d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:44.041803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fe0bc-c981-4997-9fac-0448e60fe43d None admin] RBAC: Adding query filter params (role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, user.id=e1e788b101514adb8db4fd793b954135, scope.system=all) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:44.042111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fe0bc-c981-4997-9fac-0448e60fe43d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:44.043670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fe0bc-c981-4997-9fac-0448e60fe43d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:44.059618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 164/334] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/role_assignments?user.id=e1e788b101514adb8db4fd793b954135&scope.system=all&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:45.846339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e2c2d-3f6b-49f1-9004-484d74c9a7df None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.846867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e2c2d-3f6b-49f1-9004-484d74c9a7df None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.846867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e2c2d-3f6b-49f1-9004-484d74c9a7df None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.848679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 171/335] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:45.866358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f834-5e2d-46c6-9131-60dfb6504dd8 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.866852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f834-5e2d-46c6-9131-60dfb6504dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.866954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8f834-5e2d-46c6-9131-60dfb6504dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.901414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f9b8f834-5e2d-46c6-9131-60dfb6504dd8 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:45.902637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f9b8f834-5e2d-46c6-9131-60dfb6504dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:45.926943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106821b4-a193-4966-9b4f-cd389094be91 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.927266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106821b4-a193-4966-9b4f-cd389094be91 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.927500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106821b4-a193-4966-9b4f-cd389094be91 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.928889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 172/336] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:45.934113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f095e8-de78-4a3a-9a8d-02e92ac47c7c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.934636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f095e8-de78-4a3a-9a8d-02e92ac47c7c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.935102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f095e8-de78-4a3a-9a8d-02e92ac47c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.972245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-46f095e8-de78-4a3a-9a8d-02e92ac47c7c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:45.973857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-46f095e8-de78-4a3a-9a8d-02e92ac47c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:45.995230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 165/337] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:46.036592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8f834-5e2d-46c6-9131-60dfb6504dd8 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.080853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.087056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 173/338] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:46.112320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f095e8-de78-4a3a-9a8d-02e92ac47c7c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.163743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8f834-5e2d-46c6-9131-60dfb6504dd8 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.201909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.202905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.203402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.204580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.219419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:46.225258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:46.226172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:46.226927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:46.229699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 166/339] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:46.240071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.265776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f095e8-de78-4a3a-9a8d-02e92ac47c7c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.284122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.313113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.314020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.314476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.314881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.330707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:46.335980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:46.341256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:46.342707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:46.351606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 174/340] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:46.377471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.378749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.418028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396783b-afb6-4ab3-8e6a-1644418ee3bd None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.494462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.495091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.495372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.495644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.496373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:46.496899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.498556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.516007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 167/341] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.521676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.541859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d9c90-7ccd-47c0-9b5b-912fd9011c86 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.624136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff795f7-c990-4f9b-b59d-46ef8d37a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.625122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff795f7-c990-4f9b-b59d-46ef8d37a93a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.625577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff795f7-c990-4f9b-b59d-46ef8d37a93a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.626023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff795f7-c990-4f9b-b59d-46ef8d37a93a None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.626918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff795f7-c990-4f9b-b59d-46ef8d37a93a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:46.627422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff795f7-c990-4f9b-b59d-46ef8d37a93a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.629062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff795f7-c990-4f9b-b59d-46ef8d37a93a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.646664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822af24-5df9-4e40-9a1f-e575c79efdad None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.648085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 175/342] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:46.649510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a9b28-6c21-4ff8-a520-04ab6452c034 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.650028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a9b28-6c21-4ff8-a520-04ab6452c034 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.650519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a9b28-6c21-4ff8-a520-04ab6452c034 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.652154 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 176/343] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:46.661908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725715b-7795-4727-8b9b-e52000fbc9f1 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.662481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725715b-7795-4727-8b9b-e52000fbc9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.662953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725715b-7795-4727-8b9b-e52000fbc9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.688192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc966d-fa7a-4ddc-a9bb-80500d7fcbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.689096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc966d-fa7a-4ddc-a9bb-80500d7fcbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.689543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc966d-fa7a-4ddc-a9bb-80500d7fcbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.689966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc966d-fa7a-4ddc-a9bb-80500d7fcbc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.690632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e725715b-7795-4727-8b9b-e52000fbc9f1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:46.691957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e725715b-7795-4727-8b9b-e52000fbc9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:46.706060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.745221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.771463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc966d-fa7a-4ddc-a9bb-80500d7fcbc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.773221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc966d-fa7a-4ddc-a9bb-80500d7fcbc4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.788658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 177/344] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:46.789837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725715b-7795-4727-8b9b-e52000fbc9f1 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.931868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 168/345] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:46.933240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725715b-7795-4727-8b9b-e52000fbc9f1 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.934701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc97be3-a6fa-4e83-b87f-70bfce8c1169 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.935201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc97be3-a6fa-4e83-b87f-70bfce8c1169 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.935704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc97be3-a6fa-4e83-b87f-70bfce8c1169 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.937733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 169/346] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:46.939614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc97be3-a6fa-4e83-b87f-70bfce8c1169 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.986716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.987376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.987657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.987930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] PATH_INFO: `/v3/users/701e9561643f4a20a5206fae52e3fc3d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.989152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] RBAC: Authorizing `identity:get_user(user_id=701e9561643f4a20a5206fae52e3fc3d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.990594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.992356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 178/347] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/users/701e9561643f4a20a5206fae52e3fc3d => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.993358 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.018542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.067571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.072231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc97be3-a6fa-4e83-b87f-70bfce8c1169 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.114904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718d99d-f3d7-41c5-8a8e-7c30920ff673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.115823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d99d-f3d7-41c5-8a8e-7c30920ff673 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.116265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d99d-f3d7-41c5-8a8e-7c30920ff673 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.116702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d99d-f3d7-41c5-8a8e-7c30920ff673 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.130322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718d99d-f3d7-41c5-8a8e-7c30920ff673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-b718d99d-f3d7-41c5-8a8e-7c30920ff673 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:47.135730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:47.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:47.137546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:47.142195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 170/348] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:47.143814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b251a9-915d-40da-8cb5-74419ff8892a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.144246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b251a9-915d-40da-8cb5-74419ff8892a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.144700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b251a9-915d-40da-8cb5-74419ff8892a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.146617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f1b2-6c07-467a-b6f7-2d33d14c16d0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.165519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-35b251a9-915d-40da-8cb5-74419ff8892a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:47.168836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-35b251a9-915d-40da-8cb5-74419ff8892a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:47.191231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.191961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.192519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.192954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.195014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.196408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.198171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 179/349] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:47.199059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.258134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 171/350] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:47.259395 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b251a9-915d-40da-8cb5-74419ff8892a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.295819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.321246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9053a09-6c93-40fb-990f-887c188c22f9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.340221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.369677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b251a9-915d-40da-8cb5-74419ff8892a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.373891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.373891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.374203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.374459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.375325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] RBAC: Adding query filter params (role.id=46fcb1e9e111430489c72b70f557e173, user.id=701e9561643f4a20a5206fae52e3fc3d, scope.system=all) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:47.375630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.377084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.392408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 180/351] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/role_assignments?user.id=701e9561643f4a20a5206fae52e3fc3d&scope.system=all&role.id=46fcb1e9e111430489c72b70f557e173 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:47.394599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.411594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.412422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.412802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.413174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.414038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:47.414477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.416016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.426179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 172/352] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:47.427230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.535217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b247d-7632-4753-89e7-f89ed1e21db2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.543879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.552565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c1635-1143-42a2-928f-ef8a9a1156c6 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.585539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78e7c0-43a3-47d3-85ff-7be9b0638595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.586816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78e7c0-43a3-47d3-85ff-7be9b0638595 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.587277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78e7c0-43a3-47d3-85ff-7be9b0638595 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.587728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78e7c0-43a3-47d3-85ff-7be9b0638595 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.593986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.594736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.596267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.596419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.598156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78e7c0-43a3-47d3-85ff-7be9b0638595 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.600642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78e7c0-43a3-47d3-85ff-7be9b0638595 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.603349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 181/353] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:47.605120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8990-c0c4-4b8d-94c7-8862dd209940 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.605601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8990-c0c4-4b8d-94c7-8862dd209940 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.606535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd8990-c0c4-4b8d-94c7-8862dd209940 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.608802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 182/354] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:47.609712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd8990-c0c4-4b8d-94c7-8862dd209940 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.614256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:47.620028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:47.620975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:47.621758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:47.622922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 173/355] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:47.624547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.652270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.754393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d9ccd-e0f6-4156-8576-57e2c7ea03ba None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.757543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd8990-c0c4-4b8d-94c7-8862dd209940 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.798181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af58917-83c0-477b-af0e-8e010114dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.799620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af58917-83c0-477b-af0e-8e010114dfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.799987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af58917-83c0-477b-af0e-8e010114dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.800188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af58917-83c0-477b-af0e-8e010114dfe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.801251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af58917-83c0-477b-af0e-8e010114dfe0 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=3267824b193a4a3082969dc9c0836861) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:47.801659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af58917-83c0-477b-af0e-8e010114dfe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.803130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af58917-83c0-477b-af0e-8e010114dfe0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.804849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.805479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.805765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.806237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.814665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 174/356] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:47.816196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd6737-cd61-4125-b003-3f3ee599dd84 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.816774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd6737-cd61-4125-b003-3f3ee599dd84 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.817254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd6737-cd61-4125-b003-3f3ee599dd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.837982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-cdbd6737-cd61-4125-b003-3f3ee599dd84 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:47.839633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cdbd6737-cd61-4125-b003-3f3ee599dd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:47.856909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.885134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.886622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.921750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.925331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 175/357] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:47.927101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd6737-cd61-4125-b003-3f3ee599dd84 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.010507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 183/358] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:48.012284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.070366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd6737-cd61-4125-b003-3f3ee599dd84 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.119597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b900777-06f0-4363-aee5-d3fd4f21bebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.120368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b900777-06f0-4363-aee5-d3fd4f21bebf None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.120690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b900777-06f0-4363-aee5-d3fd4f21bebf None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.120965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b900777-06f0-4363-aee5-d3fd4f21bebf None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.121699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b900777-06f0-4363-aee5-d3fd4f21bebf None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:48.122003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b900777-06f0-4363-aee5-d3fd4f21bebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.125020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b900777-06f0-4363-aee5-d3fd4f21bebf None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.134362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 176/359] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.135441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e432-3ca8-4683-9606-3fb597275f8d None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.135722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e432-3ca8-4683-9606-3fb597275f8d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.135995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e432-3ca8-4683-9606-3fb597275f8d None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.137254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 177/360] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:48.137936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e432-3ca8-4683-9606-3fb597275f8d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.149702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe751c38-b72e-45b1-a9d5-f1737da2bbb6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.160478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.187405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.195820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.196508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.196790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.197144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.216246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:48.222882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:48.223797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:48.224617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 184/361] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:48.225278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.278387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e432-3ca8-4683-9606-3fb597275f8d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.324662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f9570-d11e-4160-a0dc-6c7c13db024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.325475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f9570-d11e-4160-a0dc-6c7c13db024e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.325907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f9570-d11e-4160-a0dc-6c7c13db024e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.326799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f9570-d11e-4160-a0dc-6c7c13db024e None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.328160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f9570-d11e-4160-a0dc-6c7c13db024e None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.329764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f9570-d11e-4160-a0dc-6c7c13db024e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.333628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 178/362] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.334856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f710b6-48ff-459b-941c-805c2a4798c7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.337648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f710b6-48ff-459b-941c-805c2a4798c7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.337648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f710b6-48ff-459b-941c-805c2a4798c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.357141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-72f710b6-48ff-459b-941c-805c2a4798c7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:48.358697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-72f710b6-48ff-459b-941c-805c2a4798c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:48.362792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64fd93-0043-4b40-9585-31637c7e7d71 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.408945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.409572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.409850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.410121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.453350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 179/363] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:48.455083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f710b6-48ff-459b-941c-805c2a4798c7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.477867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.486733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.498755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.500317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.585793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f710b6-48ff-459b-941c-805c2a4798c7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.618246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 185/364] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:48.619134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.634132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.635142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.635654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.636104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.637070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:48.639169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.641003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.656109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 180/365] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.656109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.728374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14d3a2-529c-43d1-be5f-b28a76bb8f56 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.755979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.791498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376e9e8-d799-4011-8539-d080d42a572d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.795653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.796311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.796640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.796917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.801320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.801320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.808486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 186/366] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.808486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.831420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.832433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.832911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.833378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.845888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:48.852569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:48.853440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:48.854227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:48.855093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 181/367] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:48.855900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.922728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50f630-3254-4cbe-9944-b5042c8ccbec None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.967225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.967851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.968127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.968419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.969601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.971043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.977116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 187/368] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.977116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.990964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b558fd-bfb8-4fa1-aed2-7c8aa1d0b21a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.007041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.026319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.036706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.037455 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.037818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.038167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.102564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded50d25-9d81-41b8-a27d-bd1c48b2fbe1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.141802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.142726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.143001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.143278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.144063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=3267824b193a4a3082969dc9c0836861) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:49.144328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.144764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.145887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.147662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.158806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 188/369] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.159946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.268124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 182/370] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:49.268914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.280557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.297978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ccd05-bb36-400d-a331-da0554b55034 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.336880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.346454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.347240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.347524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.351711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.351711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:49.351711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.351711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.363771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 189/371] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.364776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.402776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba150f-e321-4b4f-a55d-b0b087537b85 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.456628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.457279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.457558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.457832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.459666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.461483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.464652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 183/372] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.465504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.500772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73f7af-7504-4a8c-8bc9-df2fd9360fe4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.539969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.541066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.541580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.542060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.543552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.545836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.550221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 190/373] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.551280 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.573214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.594156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.597949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed175833-bd70-41e8-98f4-8ed1addba675 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.647826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.647826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.647826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.647826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.667919 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f17c8-f36c-4ee8-9837-155f27ff0149 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.707591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.708634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.709116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.709629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.711047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=2a18415ea9674b58862ea134ee2ad684) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:49.711559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.714021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.723391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 191/374] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.729629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.751438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.753206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.861581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.884334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c32d8-3d6b-4c56-81d2-58b91f550f10 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.937211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.938176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.938601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.939019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.940421 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.943365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.945268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 192/375] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.957097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.993405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.997807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 184/376] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:50.002892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.072798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ef731-86cb-4833-935c-50397190bde1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.109502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ecec-2c8d-4cc6-9601-f47f97501140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.110118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ecec-2c8d-4cc6-9601-f47f97501140 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.110400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ecec-2c8d-4cc6-9601-f47f97501140 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.110682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ecec-2c8d-4cc6-9601-f47f97501140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.111460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213ecec-2c8d-4cc6-9601-f47f97501140 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=2a18415ea9674b58862ea134ee2ad684) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:50.111767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213ecec-2c8d-4cc6-9601-f47f97501140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.114011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213ecec-2c8d-4cc6-9601-f47f97501140 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.124518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 193/377] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:50.125995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3bc49-1292-4d6c-9c27-ed2c785a5835 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.126281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3bc49-1292-4d6c-9c27-ed2c785a5835 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.126559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3bc49-1292-4d6c-9c27-ed2c785a5835 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.128166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 194/378] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:50.138068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731261a-7718-429f-b10f-25be9947ec71 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.138887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731261a-7718-429f-b10f-25be9947ec71 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.139143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731261a-7718-429f-b10f-25be9947ec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.149122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa808db-4ae7-4ed9-a1c3-5c6486eba455 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.176693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d731261a-7718-429f-b10f-25be9947ec71 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:50.177968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d731261a-7718-429f-b10f-25be9947ec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:50.193624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.194223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.194610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.194937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] PATH_INFO: `/v3/projects/285f06e24b5441ccbaca698e1fbd1162` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.198574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.201278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] RBAC: Authorizing `identity:get_project(project_id=285f06e24b5441ccbaca698e1fbd1162)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.204335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.206905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 185/379] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/projects/285f06e24b5441ccbaca698e1fbd1162 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:50.212960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.304478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.340752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2b6db-fad8-46cc-9bda-2a590b16283a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.371532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 195/380] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:50.381867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.382598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.382981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.383327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] PATH_INFO: `/v3/groups/e763827500484ab492d8058f0b538eee` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.390378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] RBAC: Authorizing `identity:get_group(group_id=e763827500484ab492d8058f0b538eee)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.391984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.394104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 186/381] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/groups/e763827500484ab492d8058f0b538eee => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:50.399965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731261a-7718-429f-b10f-25be9947ec71 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.407947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.571276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731261a-7718-429f-b10f-25be9947ec71 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.611557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de68e8-885c-475b-88dd-d24da7abc3cf None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.649742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.656413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.657216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.657523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.657805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.670329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:50.676129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:50.677064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:50.677849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:50.682427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 187/382] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:50.683785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.697109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.697754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.698034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.698304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.699105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] RBAC: Adding query filter params (group.id=e763827500484ab492d8058f0b538eee, role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=285f06e24b5441ccbaca698e1fbd1162) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:50.699411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.701415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.722323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 196/383] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/role_assignments?group.id=e763827500484ab492d8058f0b538eee&scope.project.id=285f06e24b5441ccbaca698e1fbd1162&role.id=7912a7addff34d9582eb2933d7110ebc => generated 260 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:50.800544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1b63a-0739-47b1-be1c-20d6ecdadacb None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.838044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.838614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.838903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.839182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.839926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:50.840236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.842251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.854697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 188/384] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:50.857798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.870522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.980859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10652916-ac55-410d-849e-a7be8bfab3d1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.019118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.019782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.020065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.020343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.021541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.023000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.028613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 197/385] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:51.030928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.094821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.141537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3308b6b-c61e-4a76-b70a-43bd2fb7b46a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.177178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fa007-dc78-4c74-af9f-8224c52d028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.177909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fa007-dc78-4c74-af9f-8224c52d028d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.178146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fa007-dc78-4c74-af9f-8224c52d028d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.178436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fa007-dc78-4c74-af9f-8224c52d028d None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.179731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fa007-dc78-4c74-af9f-8224c52d028d None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.181191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fa007-dc78-4c74-af9f-8224c52d028d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.185441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 189/386] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:51 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:51.189557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.283230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.307474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62337ad9-f61c-4128-b42b-ee27f6a6fee2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.344029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaee57-8b3a-4c21-a950-32af0518575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.344745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaee57-8b3a-4c21-a950-32af0518575a None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.345089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaee57-8b3a-4c21-a950-32af0518575a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.345311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaee57-8b3a-4c21-a950-32af0518575a None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.347397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaee57-8b3a-4c21-a950-32af0518575a None admin] RBAC: Authorizing `identity:create_grant(project_id=3267824b193a4a3082969dc9c0836861, group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.348930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaee57-8b3a-4c21-a950-32af0518575a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.434073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 198/387] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 27 19:54:51 2022] PUT /identity/v3/projects/3267824b193a4a3082969dc9c0836861/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:51.570686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33650386-44e5-41a8-bf51-97443f96309a None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.571043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33650386-44e5-41a8-bf51-97443f96309a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.571281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33650386-44e5-41a8-bf51-97443f96309a None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.572978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 190/388] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:51.578646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a1c1a-03ff-4ed2-ba69-20a5553d4589 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.578938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a1c1a-03ff-4ed2-ba69-20a5553d4589 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.579219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a1c1a-03ff-4ed2-ba69-20a5553d4589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.599085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bf8a1c1a-03ff-4ed2-ba69-20a5553d4589 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:51.600236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bf8a1c1a-03ff-4ed2-ba69-20a5553d4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:51.684180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 199/389] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:51.709252 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33650386-44e5-41a8-bf51-97443f96309a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.836919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33650386-44e5-41a8-bf51-97443f96309a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.871890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.885453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd63f4-ccf0-40ee-a546-b149884f65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.886109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd63f4-ccf0-40ee-a546-b149884f65b6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.886395 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd63f4-ccf0-40ee-a546-b149884f65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.890770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd63f4-ccf0-40ee-a546-b149884f65b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.902993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd63f4-ccf0-40ee-a546-b149884f65b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-68fd63f4-ccf0-40ee-a546-b149884f65b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:51.908137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:51.909022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:51.909887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 191/390] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:51.915672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a1c1a-03ff-4ed2-ba69-20a5553d4589 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.001713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.052476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a1c1a-03ff-4ed2-ba69-20a5553d4589 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.101306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157f0e2-8330-464e-80ea-4556d3570399 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.101619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157f0e2-8330-464e-80ea-4556d3570399 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.101861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157f0e2-8330-464e-80ea-4556d3570399 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.103352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 192/391] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:52.105273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.105904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.107056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.108449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b4a0-7bcc-4df1-80ae-b3fa8e990027 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.108736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b4a0-7bcc-4df1-80ae-b3fa8e990027 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.109010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b4a0-7bcc-4df1-80ae-b3fa8e990027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.113972 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.114708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:52.115020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.116415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.139391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-0733b4a0-7bcc-4df1-80ae-b3fa8e990027 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:52.139391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0733b4a0-7bcc-4df1-80ae-b3fa8e990027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:52.140680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 200/392] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:52.151214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.233001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 193/393] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:52.266984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733b4a0-7bcc-4df1-80ae-b3fa8e990027 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.348402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967431b-8266-4b0a-bcaf-0de2f0204d9f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.404838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.413599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.414260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.414545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.414822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.417215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.418664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.429194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 201/394] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:52.430049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.491195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733b4a0-7bcc-4df1-80ae-b3fa8e990027 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.522976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.556395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a187eaa-80c3-43ee-8e2a-6f092540ff85 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.592785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.593521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.593980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.594404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.595730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.597449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.599709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 202/395] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:52.604817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.605721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.606349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.607164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.607437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.644983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:52.650064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:52.651134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:52.652187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 194/396] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:52.661324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.692769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.717722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd242f47-729c-4c66-8f8b-75e0c46d0f21 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.754900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.755825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.756257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.756878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.760217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] RBAC: Authorizing `identity:create_grant(project_id=3267824b193a4a3082969dc9c0836861, group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.761974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.836536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177a458-569c-47b1-889c-d6868f9dbecf None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.855854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 203/397] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 27 19:54:52 2022] PUT /identity/v3/projects/3267824b193a4a3082969dc9c0836861/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:52.884867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199b01-41a5-4494-b14a-565bfcf6d817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.885519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199b01-41a5-4494-b14a-565bfcf6d817 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.885803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199b01-41a5-4494-b14a-565bfcf6d817 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.886082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199b01-41a5-4494-b14a-565bfcf6d817 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.886825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199b01-41a5-4494-b14a-565bfcf6d817 None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:52.887135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199b01-41a5-4494-b14a-565bfcf6d817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.889221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199b01-41a5-4494-b14a-565bfcf6d817 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.903230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 195/398] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:52.909466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.059395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cd923-a30f-4d76-a1b5-6cfc73865f86 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.081022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d383e-55c6-4436-9963-f9b46664df7a None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.081638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d383e-55c6-4436-9963-f9b46664df7a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.082108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d383e-55c6-4436-9963-f9b46664df7a None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.083883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 196/399] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:53.087671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.089046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c4baf-d95d-45b4-a1ee-10b37a3477b9 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.089548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c4baf-d95d-45b4-a1ee-10b37a3477b9 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.090014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c4baf-d95d-45b4-a1ee-10b37a3477b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.097932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.098592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.098876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.099158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.101406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.102902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.104696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 204/400] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:53.112225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.113179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d26c4baf-d95d-45b4-a1ee-10b37a3477b9 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:53.114576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d26c4baf-d95d-45b4-a1ee-10b37a3477b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:53.144938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.191324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 197/401] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:53.216241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c4baf-d95d-45b4-a1ee-10b37a3477b9 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.243999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f65c8d-a1ee-455f-a477-1e7e8793b8f5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.296011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.299057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.299565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.300109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.302098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.306516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.309928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 205/402] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:53.315689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.355585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c4baf-d95d-45b4-a1ee-10b37a3477b9 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.393040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f128-ea2f-435a-8499-2cc44ab0aaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.393682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f128-ea2f-435a-8499-2cc44ab0aaad None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.393976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f128-ea2f-435a-8499-2cc44ab0aaad None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.394238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f128-ea2f-435a-8499-2cc44ab0aaad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.396611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.405268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79f128-ea2f-435a-8499-2cc44ab0aaad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-5f79f128-ea2f-435a-8499-2cc44ab0aaad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:53.410462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:53.417172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 198/403] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:53.417964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f44cd-8387-477c-9306-0e04a27312b0 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.418829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f44cd-8387-477c-9306-0e04a27312b0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.418829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f44cd-8387-477c-9306-0e04a27312b0 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.419528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 199/404] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:53.421104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bf871-0960-40ab-b255-2a68abb93b39 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.421226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bf871-0960-40ab-b255-2a68abb93b39 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.421440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bf871-0960-40ab-b255-2a68abb93b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.443421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b71bf871-0960-40ab-b255-2a68abb93b39 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:53.445462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b71bf871-0960-40ab-b255-2a68abb93b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:53.457944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceaa129-7431-4415-a01c-ac6927693eba None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.470592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.504936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.505622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.505898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.506468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.509420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a18415ea9674b58862ea134ee2ad684, group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.511504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.529775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 200/405] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:53.530930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bf871-0960-40ab-b255-2a68abb93b39 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.598877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 206/406] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 27 19:54:53 2022] PUT /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:53.604605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.692882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bf871-0960-40ab-b255-2a68abb93b39 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.750891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.753460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.753761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.754039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.754806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:53.755116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.756575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.778347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 201/407] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:53.794668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.816822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78582-522d-4f75-8e28-758e84d8efc6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.820685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.849075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.868182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.869672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.870183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.870658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.884038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:53.893599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:53.894483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:53.895262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 207/408] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:53.900499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.952245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3673b-1be3-48eb-867c-958f61cc321f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.022817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.023464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.023744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.024021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.025210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.026650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.029256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 202/409] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:54.034432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.085286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfee09-20f7-4e2e-ae1a-35918c3f524a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.128665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16191cee-d81e-49aa-bb99-9c787f2feaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.128665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191cee-d81e-49aa-bb99-9c787f2feaed None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.128665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191cee-d81e-49aa-bb99-9c787f2feaed None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.128665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191cee-d81e-49aa-bb99-9c787f2feaed None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.129039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191cee-d81e-49aa-bb99-9c787f2feaed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:54.129039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191cee-d81e-49aa-bb99-9c787f2feaed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.129158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191cee-d81e-49aa-bb99-9c787f2feaed None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.142654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 208/410] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:54.143991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600334a0-deed-461f-8f0a-669f6664f0d9 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.144279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600334a0-deed-461f-8f0a-669f6664f0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.144583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600334a0-deed-461f-8f0a-669f6664f0d9 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.145849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 209/411] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:54.146992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600334a0-deed-461f-8f0a-669f6664f0d9 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.187570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.192864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dbb98-f894-4353-8ab5-1305e27e3161 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.201756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.241471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58a7e8-620d-4208-9411-bf4c4397920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.242140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a7e8-620d-4208-9411-bf4c4397920c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.242738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a7e8-620d-4208-9411-bf4c4397920c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.243076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a7e8-620d-4208-9411-bf4c4397920c None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.244527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58a7e8-620d-4208-9411-bf4c4397920c None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.246163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58a7e8-620d-4208-9411-bf4c4397920c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.248739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 203/412] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:54.250974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992398b-11af-46e9-9089-14d12e39e17d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.251260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992398b-11af-46e9-9089-14d12e39e17d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.251942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992398b-11af-46e9-9089-14d12e39e17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.269485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600334a0-deed-461f-8f0a-669f6664f0d9 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.274597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c992398b-11af-46e9-9089-14d12e39e17d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:54.275800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c992398b-11af-46e9-9089-14d12e39e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:54.314848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.315908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.316195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.316777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] PATH_INFO: `/v3/groups/e763827500484ab492d8058f0b538eee` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.317951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] RBAC: Authorizing `identity:get_group(group_id=e763827500484ab492d8058f0b538eee)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.319390 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.321455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 210/413] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/groups/e763827500484ab492d8058f0b538eee => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:54.322984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.360125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 204/414] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:54.361285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992398b-11af-46e9-9089-14d12e39e17d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.463576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.475995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5fdb5-48ce-477c-a23a-01f0c6ad1d1d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.483377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992398b-11af-46e9-9089-14d12e39e17d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.489390 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.519698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.520356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.520667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.520946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.522332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.523081 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.523588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.523961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] PATH_INFO: `/v3/projects/285f06e24b5441ccbaca698e1fbd1162` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.524166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a18415ea9674b58862ea134ee2ad684, group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.525377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] RBAC: Authorizing `identity:get_project(project_id=285f06e24b5441ccbaca698e1fbd1162)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.525663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.527009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.534352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 205/415] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/projects/285f06e24b5441ccbaca698e1fbd1162 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:54.535325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.589617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 211/416] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 27 19:54:54 2022] PUT /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:54.591770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.658701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45834-e346-4eea-a7d1-062d213cd4fc None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.725968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9f9ec-778f-42f7-9781-d78ec21ee8d9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.728823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.729493 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.729771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.730042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.757456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:54.763437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:54.765645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:54.767551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:54.769001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.769001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.769001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.769001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] PATH_INFO: `/v3/projects/285f06e24b5441ccbaca698e1fbd1162/groups/e763827500484ab492d8058f0b538eee/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.769562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 206/417] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:54.773891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.774139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] RBAC: Authorizing `identity:create_grant(project_id=285f06e24b5441ccbaca698e1fbd1162, group_id=e763827500484ab492d8058f0b538eee, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.775796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.797366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.845291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 212/418] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Jul 27 19:54:54 2022] PUT /identity/v3/projects/285f06e24b5441ccbaca698e1fbd1162/groups/e763827500484ab492d8058f0b538eee/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:54.909440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ace59-7d46-436a-82cb-306fa2cc814b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.964421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8520e-4119-4c1c-95cc-ddad4572dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.964421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8520e-4119-4c1c-95cc-ddad4572dd47 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.964421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8520e-4119-4c1c-95cc-ddad4572dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.964421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8520e-4119-4c1c-95cc-ddad4572dd47 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.964832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8520e-4119-4c1c-95cc-ddad4572dd47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:54.964832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8520e-4119-4c1c-95cc-ddad4572dd47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.964832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8520e-4119-4c1c-95cc-ddad4572dd47 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.976364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 207/419] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:54.982574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.037714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.138860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3a766-ea25-4394-87ea-afcb89521913 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.164855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a5702-da82-4607-a8b6-ab753ccb1242 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.165133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a5702-da82-4607-a8b6-ab753ccb1242 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.165417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a5702-da82-4607-a8b6-ab753ccb1242 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.167025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 208/420] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:55.173731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce56a7a-2513-451d-9438-592d40f8de43 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.174892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce56a7a-2513-451d-9438-592d40f8de43 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.175527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce56a7a-2513-451d-9438-592d40f8de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.196718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.197712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.198194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.198724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.212315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fce56a7a-2513-451d-9438-592d40f8de43 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:55.216145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fce56a7a-2513-451d-9438-592d40f8de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:55.298005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.300135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.352594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 209/421] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:55.360792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.376867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce56a7a-2513-451d-9438-592d40f8de43 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.379067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.441757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 213/422] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:55.450180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.532441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce56a7a-2513-451d-9438-592d40f8de43 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.569901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.570468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.570749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.571026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.582840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:55.587820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:55.588803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:55.589627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 210/423] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:55.600111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.631176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1f7da-eb0e-4f23-86c5-eb8c6a4fb3a9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.674849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.687542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.704598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.705253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.705528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.705796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.706986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.708428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.710371 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 214/424] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:55.719295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943245bc-00ca-4c4a-a19f-21571bf5569c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.727067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.768077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.768725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.769009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.769282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.770122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:55.770527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.771904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.792763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 211/425] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:55.798782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.884585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd62e3-adae-4a11-83fe-dbc17d9d7fb6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.949066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.949812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.950100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.950375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.951581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.953106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.955015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 215/426] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:55.960409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.006755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af6d54-e49b-4a0b-9333-cf2f0f1e774c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.049491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.071513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.072145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.072476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.072713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.072827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.123974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c920c-8934-4223-9f6a-d0ac3713188f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.164991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.167016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.170217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a684e-510f-483e-95ae-26ad6f48cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.170926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a684e-510f-483e-95ae-26ad6f48cee3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.172205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a684e-510f-483e-95ae-26ad6f48cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.172857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a684e-510f-483e-95ae-26ad6f48cee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.173730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a684e-510f-483e-95ae-26ad6f48cee3 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=3267824b193a4a3082969dc9c0836861) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:56.174118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a684e-510f-483e-95ae-26ad6f48cee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.175631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a684e-510f-483e-95ae-26ad6f48cee3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.188498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 216/427] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:56.188498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa91d0-4f2a-4571-bd84-8090da1160fa None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.188498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa91d0-4f2a-4571-bd84-8090da1160fa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.188498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa91d0-4f2a-4571-bd84-8090da1160fa None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.188498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 217/428] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:56.202329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c41ac9-72df-45b2-8383-2991385472ab None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.202665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c41ac9-72df-45b2-8383-2991385472ab None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.202920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c41ac9-72df-45b2-8383-2991385472ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.247442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f1c41ac9-72df-45b2-8383-2991385472ab None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:56.248744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f1c41ac9-72df-45b2-8383-2991385472ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:56.345550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 218/429] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:56.372342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 212/430] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:56.377613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c41ac9-72df-45b2-8383-2991385472ab None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.396997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.431170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.465516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.518158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c41ac9-72df-45b2-8383-2991385472ab None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.534366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1b820-f50e-4583-8be2-63cdc1f9b8e3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.569913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.571014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.571393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.571768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] PATH_INFO: `/v3/projects/3267824b193a4a3082969dc9c0836861` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.573796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] RBAC: Authorizing `identity:get_project(project_id=3267824b193a4a3082969dc9c0836861)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.576515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.579413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 219/431] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/projects/3267824b193a4a3082969dc9c0836861 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:56.582015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.582658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.582937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.583200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.584300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.595104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:56.600288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:56.601209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:56.601977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:54:56.610201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 213/432] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:56.611373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.718431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.731778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f8078-2e44-4f1c-8414-363b2b4e7350 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.744311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69e0de-d6b9-4c43-822c-9fd3ea263e8a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.770916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.780986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.784620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.784620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.784620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.784852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.784949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] RBAC: Adding query filter params (name=member) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:56.785018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.785088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.785159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.785159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.785293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.786066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.787481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.792053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 220/433] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:56.800659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.802469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 214/434] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:56.810483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.920503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10f73e-5abb-4497-9559-8a4c26f3c0ab None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.943847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620fec-03c2-45f9-a4dc-98f576fd901e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.959329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca0b1-6bb0-4f3e-92e9-8bd8e9197fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.960223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca0b1-6bb0-4f3e-92e9-8bd8e9197fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.960682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca0b1-6bb0-4f3e-92e9-8bd8e9197fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.961099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca0b1-6bb0-4f3e-92e9-8bd8e9197fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.962123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ca0b1-6bb0-4f3e-92e9-8bd8e9197fe5 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=3267824b193a4a3082969dc9c0836861) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:56.962598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ca0b1-6bb0-4f3e-92e9-8bd8e9197fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.964035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ca0b1-6bb0-4f3e-92e9-8bd8e9197fe5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.971904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 221/435] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=3267824b193a4a3082969dc9c0836861&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:56.986314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.986957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.987307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.987587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.006469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.013779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87996579-befe-4c07-b1d7-5bfd8eb04924 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.014082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87996579-befe-4c07-b1d7-5bfd8eb04924 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.014326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87996579-befe-4c07-b1d7-5bfd8eb04924 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.015788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 222/436] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:57.021535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e419e-c333-485d-8b05-bcd1995d3b8d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.021835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e419e-c333-485d-8b05-bcd1995d3b8d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.022066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e419e-c333-485d-8b05-bcd1995d3b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.054973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-fe1e419e-c333-485d-8b05-bcd1995d3b8d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:57.056840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fe1e419e-c333-485d-8b05-bcd1995d3b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:57.073394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.075292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.143955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.157708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 223/437] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:57.181206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e419e-c333-485d-8b05-bcd1995d3b8d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.200100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 215/438] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:57.207643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.274343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.302442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e419e-c333-485d-8b05-bcd1995d3b8d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.342065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e1cd2-a872-423b-96ca-e71120b475ec None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.347310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4ac0e-67b9-4cf3-80bd-4a486a1d89c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.348155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4ac0e-67b9-4cf3-80bd-4a486a1d89c5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.348486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4ac0e-67b9-4cf3-80bd-4a486a1d89c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.348768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4ac0e-67b9-4cf3-80bd-4a486a1d89c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.360687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4ac0e-67b9-4cf3-80bd-4a486a1d89c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-84e4ac0e-67b9-4cf3-80bd-4a486a1d89c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:57.365715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:57.367836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 224/439] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:57.368667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643abe9-2785-4f1a-ad05-cb6861530f0f None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.369359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643abe9-2785-4f1a-ad05-cb6861530f0f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.369359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643abe9-2785-4f1a-ad05-cb6861530f0f None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.371582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 225/440] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:54:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:57.373527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643abe9-2785-4f1a-ad05-cb6861530f0f None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.384907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5ecd4-033a-49a2-aca5-29c224dbbed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.385553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5ecd4-033a-49a2-aca5-29c224dbbed9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.385834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5ecd4-033a-49a2-aca5-29c224dbbed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.386277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5ecd4-033a-49a2-aca5-29c224dbbed9 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.388742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5ecd4-033a-49a2-aca5-29c224dbbed9 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.389503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5ecd4-033a-49a2-aca5-29c224dbbed9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.391185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 216/441] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.392302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127fa64-e9c4-4ea4-92f1-6714aa4090eb None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.393573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127fa64-e9c4-4ea4-92f1-6714aa4090eb None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.393804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127fa64-e9c4-4ea4-92f1-6714aa4090eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.416352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2127fa64-e9c4-4ea4-92f1-6714aa4090eb None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:57.418317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2127fa64-e9c4-4ea4-92f1-6714aa4090eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:57.433122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.483679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643abe9-2785-4f1a-ad05-cb6861530f0f None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.485480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 217/442] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:57.487075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127fa64-e9c4-4ea4-92f1-6714aa4090eb None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.519707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.520328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.520636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.520911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.521623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:57.521927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.523288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.531765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 226/443] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.532526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.539393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.592583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127fa64-e9c4-4ea4-92f1-6714aa4090eb None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.625919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.626554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.626834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.627114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.628257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.629060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88669d1c-20ab-4a54-8083-84200a6ca01e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.629749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.631348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 218/444] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.632097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.642040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.662529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.666765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.666765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.666765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.674426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:57.679886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:57.680871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:54:57.681649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 227/445] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:57.682318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.738035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e82c56-bbd7-4651-820d-ef0c0b61f15f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.751180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.772361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.772989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.773265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.773535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.779774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793c49f-3318-4188-8fa2-d20fb10c9809 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.812188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.812759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.813044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.813321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.814111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7, scope.project.id=2a18415ea9674b58862ea134ee2ad684) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:57.814421 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.815851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.821623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 228/446] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.822404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.854204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.855857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.861930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.936757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488091a-5f92-43e7-8b93-d0b055e514a3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.975316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 219/447] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:57.975500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.975700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.975984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.976109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.976269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.977039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:57.977963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.979745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.989429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 229/448] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.990614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.994957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.087296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdb874-af78-4f16-a9d3-2e4d334c5ec8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.113420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.117642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05622ee-8728-4407-ab8a-dfb5de3af834 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.131110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.131761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.132044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.132322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] PATH_INFO: `/v3/projects/2a18415ea9674b58862ea134ee2ad684` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.134685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] RBAC: Authorizing `identity:get_project(project_id=2a18415ea9674b58862ea134ee2ad684)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.136167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.138055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 220/449] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/projects/2a18415ea9674b58862ea134ee2ad684 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:58.145949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.161823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.162765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.163101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.163384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.231562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.238327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.239860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.251298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160491ab-c15a-48f5-989d-a83ff69fee75 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.284758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.285392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.285673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.285948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] PATH_INFO: `/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.287118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] RBAC: Authorizing `identity:get_group(group_id=7e9682dcbc7648f0b9a0e47a9ec6fbda)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.288588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.290197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 221/450] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/groups/7e9682dcbc7648f0b9a0e47a9ec6fbda => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:58.296561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.334482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.342953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 230/451] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:58.346675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.396284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93075a2-85a5-40b5-81ff-1a87fc0420b9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.429594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.430189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.430452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.430719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.431500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] RBAC: Adding query filter params (group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda, role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=2a18415ea9674b58862ea134ee2ad684) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:58.431803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.433785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.439362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 222/452] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/role_assignments?group.id=7e9682dcbc7648f0b9a0e47a9ec6fbda&scope.project.id=2a18415ea9674b58862ea134ee2ad684&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:58.443495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d6365-c57d-4b7e-bb59-20c658b93ede None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.480679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.481376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.481658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.481933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] PATH_INFO: `/v3/projects/285f06e24b5441ccbaca698e1fbd1162` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.483164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] RBAC: Authorizing `identity:get_project(project_id=285f06e24b5441ccbaca698e1fbd1162)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.484677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.486434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 231/453] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/projects/285f06e24b5441ccbaca698e1fbd1162 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:58.492190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.508244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.598641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f9d6-c586-4e59-84af-8f120bf2ca57 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.632850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725c016-ffb2-4cd7-ac2d-844a13b46ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.633491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725c016-ffb2-4cd7-ac2d-844a13b46ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.633781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725c016-ffb2-4cd7-ac2d-844a13b46ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.634053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725c016-ffb2-4cd7-ac2d-844a13b46ab6 None admin] PATH_INFO: `/v3/groups/e763827500484ab492d8058f0b538eee` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.635221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725c016-ffb2-4cd7-ac2d-844a13b46ab6 None admin] RBAC: Authorizing `identity:get_group(group_id=e763827500484ab492d8058f0b538eee)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.636726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725c016-ffb2-4cd7-ac2d-844a13b46ab6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.638414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 223/454] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/groups/e763827500484ab492d8058f0b538eee => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:58.644446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.714105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.748833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa163f2f-7532-4db5-9f71-0aed038527f7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.784211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7e40-93c7-4473-9b6a-3d159eca392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.784880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7e40-93c7-4473-9b6a-3d159eca392f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.785161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7e40-93c7-4473-9b6a-3d159eca392f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.785430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7e40-93c7-4473-9b6a-3d159eca392f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.786214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a7e40-93c7-4473-9b6a-3d159eca392f None admin] RBAC: Adding query filter params (group.id=e763827500484ab492d8058f0b538eee, role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=285f06e24b5441ccbaca698e1fbd1162) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:58.786519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a7e40-93c7-4473-9b6a-3d159eca392f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.787934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a7e40-93c7-4473-9b6a-3d159eca392f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.794236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 232/455] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/role_assignments?group.id=e763827500484ab492d8058f0b538eee&scope.project.id=285f06e24b5441ccbaca698e1fbd1162&role.id=7912a7addff34d9582eb2933d7110ebc => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:01.246374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86d175-841b-4aa1-80ed-1ac1bd109245 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.246888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86d175-841b-4aa1-80ed-1ac1bd109245 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.246963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86d175-841b-4aa1-80ed-1ac1bd109245 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.248526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 224/456] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:01.253912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f03fc7-9130-4051-afbb-1f2b6f2c5641 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.254412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f03fc7-9130-4051-afbb-1f2b6f2c5641 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.254505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f03fc7-9130-4051-afbb-1f2b6f2c5641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.277512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-15f03fc7-9130-4051-afbb-1f2b6f2c5641 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:01.278720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-15f03fc7-9130-4051-afbb-1f2b6f2c5641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:01.355847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 233/457] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:01.392429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e484f5-fb0a-4606-8496-e284a316a1f4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.392718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e484f5-fb0a-4606-8496-e284a316a1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.392999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e484f5-fb0a-4606-8496-e284a316a1f4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.394658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 225/458] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:01.399575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f03fc7-9130-4051-afbb-1f2b6f2c5641 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:01.399932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cacf3d-c601-4cc0-9a68-b0d0e5a127b3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.400216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cacf3d-c601-4cc0-9a68-b0d0e5a127b3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.400517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cacf3d-c601-4cc0-9a68-b0d0e5a127b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.429243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-31cacf3d-c601-4cc0-9a68-b0d0e5a127b3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:01.430433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-31cacf3d-c601-4cc0-9a68-b0d0e5a127b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:01.502104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:01.540553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 226/459] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:01.542031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d03b4-084e-4bac-acfd-afb41f9dacb6 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.542324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d03b4-084e-4bac-acfd-afb41f9dacb6 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.542606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d03b4-084e-4bac-acfd-afb41f9dacb6 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.545422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 227/460] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:01.550875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73867e-50bf-4906-b307-0d3f76fa0d85 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.551156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73867e-50bf-4906-b307-0d3f76fa0d85 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.551434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73867e-50bf-4906-b307-0d3f76fa0d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.570268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f03fc7-9130-4051-afbb-1f2b6f2c5641 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:01.572606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9f73867e-50bf-4906-b307-0d3f76fa0d85 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:01.574637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9f73867e-50bf-4906-b307-0d3f76fa0d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:01.577881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:01.611360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:01.612414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.612922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.613416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.624960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:01.626682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:01.643217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:01.644137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:01.645016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:01.645016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:01.645016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:01.645016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:01.652618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 228/462] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:01.652618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73867e-50bf-4906-b307-0d3f76fa0d85 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:01.652910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 234/462] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:01.657255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:01.758540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73867e-50bf-4906-b307-0d3f76fa0d85 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:01.790302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe47574-d75b-48c8-979d-a33b01a19787 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:01.796250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:01.797296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.797680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.798080 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.805685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:01.807002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:01.810151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:01.811845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:01.827924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:01.828998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:01.829849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:01.829849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:01.829849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:01.829849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:01.834813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 229/463] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:01.834813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:01.837429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:01.838295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.838969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.839424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.840319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:01.840853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:01.842393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:01.856130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 235/464] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:01.857387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:01.943076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dbc42-08ff-489b-9aaf-7e9d26f1bb8e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:01.960674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e906c8-7eab-46d6-904b-fc45b9499df8 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:01.979845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:01.980617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.980977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.981353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.992697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:01.994257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:01.998830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:01.999673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.000151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.000652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.001528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:02.002004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.003541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:02.010403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:02.011778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:02.016911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:02.016911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:02.016911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:02.016911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:02.016911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 230/465] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:02.016911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.024023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 236/466] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:02.029959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.063696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.068763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.122120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bef8e-9550-4a4f-a9cb-a465de765903 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.138875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17663c0-e1ec-463e-ae1a-0ff5072eebb8 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.158576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede12bf4-734e-423b-8f60-fbb03e3842ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.159224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede12bf4-734e-423b-8f60-fbb03e3842ed None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.159507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede12bf4-734e-423b-8f60-fbb03e3842ed None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.159785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede12bf4-734e-423b-8f60-fbb03e3842ed None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.160502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede12bf4-734e-423b-8f60-fbb03e3842ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.161942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede12bf4-734e-423b-8f60-fbb03e3842ed None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.180740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.181372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.181650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.181928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.183143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:02.188281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.188281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.197211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 237/467] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:02.197934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.284508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 231/468] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:02.284508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dddff5-bf24-494f-866d-94c84e2baf27 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.284508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dddff5-bf24-494f-866d-94c84e2baf27 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.284508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dddff5-bf24-494f-866d-94c84e2baf27 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.284508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 232/469] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:02.284508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dddff5-bf24-494f-866d-94c84e2baf27 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.299447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.309803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313dceaf-04a5-4b8b-9e97-65f7821384f5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.357805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302da523-6b12-4143-865d-827932986b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.358492 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302da523-6b12-4143-865d-827932986b5d None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.358779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302da523-6b12-4143-865d-827932986b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.359054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302da523-6b12-4143-865d-827932986b5d None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.359764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302da523-6b12-4143-865d-827932986b5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.361985 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302da523-6b12-4143-865d-827932986b5d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.429717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 238/470] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:02.431717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.433144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e91f7-8e50-4fa0-b036-35f22f3f185c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.433523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e91f7-8e50-4fa0-b036-35f22f3f185c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.433808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e91f7-8e50-4fa0-b036-35f22f3f185c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.438321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dddff5-bf24-494f-866d-94c84e2baf27 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.457732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7f5e91f7-8e50-4fa0-b036-35f22f3f185c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:02.459353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7f5e91f7-8e50-4fa0-b036-35f22f3f185c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:02.485499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdc54c-9d87-48bd-a14b-5ae02a26d7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.486150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdc54c-9d87-48bd-a14b-5ae02a26d7d8 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.486485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdc54c-9d87-48bd-a14b-5ae02a26d7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.488056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdc54c-9d87-48bd-a14b-5ae02a26d7d8 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.488414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdc54c-9d87-48bd-a14b-5ae02a26d7d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.490041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdc54c-9d87-48bd-a14b-5ae02a26d7d8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.565520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 239/471] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:02.588308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 233/472] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/users => generated 248 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:02.594334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e91f7-8e50-4fa0-b036-35f22f3f185c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.605211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c320e8-5113-4f81-9f8e-5bcafb3dca6e None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.605488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c320e8-5113-4f81-9f8e-5bcafb3dca6e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.605786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c320e8-5113-4f81-9f8e-5bcafb3dca6e None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.607162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 234/473] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:02.613321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a0db2-1d4f-4543-b16b-a70d7ccfab9b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.613625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a0db2-1d4f-4543-b16b-a70d7ccfab9b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.614732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a0db2-1d4f-4543-b16b-a70d7ccfab9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.637851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-8d7a0db2-1d4f-4543-b16b-a70d7ccfab9b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:02.639438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8d7a0db2-1d4f-4543-b16b-a70d7ccfab9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:02.700090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.725742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e91f7-8e50-4fa0-b036-35f22f3f185c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.739427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 235/474] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:02.772781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.774994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.774994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.775150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a0db2-1d4f-4543-b16b-a70d7ccfab9b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.775249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.787528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.789878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:02.812983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:02.813881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:02.814701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:02.814701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:02.814701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:02.814701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:02.816547 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 240/475] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:02.820259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.894387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.913128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a0db2-1d4f-4543-b16b-a70d7ccfab9b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.944858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035da79-202b-470b-ad34-71f5372abe7f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.962295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.963587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.964446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.964928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.966225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.967802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.984781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.992802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.993674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.994178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.995213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.996196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:02.996748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.998137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.002187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.sql.core [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 19:55:03.002187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 19:55:03.002187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [parameters: {'id': 'a6918930c82e422db7b81bcac130baec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 19:55:03.002187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80335) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:03.026566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:03.027485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application yield resource Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise self.value Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application session.commit() Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 19:55:03.028414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.session.flush() Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application compat.raise_( Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 19:55:03.029282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:03.031481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:03.033828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [parameters: {'id': 'a6918930c82e422db7b81bcac130baec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:03.035801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 19:55:03.040576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:03.041570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 241/476] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:03.041663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 236/477] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/roles => generated 144 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:03.041731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.041805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.152319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb966460-28e4-4463-a52f-704ccafe46b7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.235143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bab2c-4ad0-4e67-9e6c-8d4620e10aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.236063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bab2c-4ad0-4e67-9e6c-8d4620e10aeb None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.236560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bab2c-4ad0-4e67-9e6c-8d4620e10aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.237003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bab2c-4ad0-4e67-9e6c-8d4620e10aeb None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.237842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bab2c-4ad0-4e67-9e6c-8d4620e10aeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:03.239449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bab2c-4ad0-4e67-9e6c-8d4620e10aeb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.255316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a30ca-20f3-48cc-9aac-80c48f4fc92d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.292528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.293219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.293505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.293788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.304793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:03.310615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:03.311499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:03.312261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 237/478] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:03.319034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.337759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.357670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 242/479] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:03 2022] POST /identity/v3/users => generated 249 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:03.464181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbb301-15a7-4536-b0c2-da352d5291c8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.508973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42a49a-46b6-4810-9919-b54facf8cd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.510032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42a49a-46b6-4810-9919-b54facf8cd2e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.510555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42a49a-46b6-4810-9919-b54facf8cd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.511031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42a49a-46b6-4810-9919-b54facf8cd2e None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.511957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42a49a-46b6-4810-9919-b54facf8cd2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:03.512482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42a49a-46b6-4810-9919-b54facf8cd2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:03.514072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42a49a-46b6-4810-9919-b54facf8cd2e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.523576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 238/480] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:04.842035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be25f6c-f453-4f6f-8efd-1287b6343916 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.842548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be25f6c-f453-4f6f-8efd-1287b6343916 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.842655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be25f6c-f453-4f6f-8efd-1287b6343916 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.844299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 243/481] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:04.849373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f56f-f807-487f-b099-93cbcd96750b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.849775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f56f-f807-487f-b099-93cbcd96750b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.849941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f56f-f807-487f-b099-93cbcd96750b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.874182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7306f56f-f807-487f-b099-93cbcd96750b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:04.875377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7306f56f-f807-487f-b099-93cbcd96750b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:04.975097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 239/482] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:05.014967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be25f6c-f453-4f6f-8efd-1287b6343916 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.038953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c354b3-d3e3-4c42-b2cc-a2525103d84c None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.039292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c354b3-d3e3-4c42-b2cc-a2525103d84c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.039512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c354b3-d3e3-4c42-b2cc-a2525103d84c None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.041013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 240/483] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:05.048019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2867e1b-31ef-40c7-a141-66b7c0168a29 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.048692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2867e1b-31ef-40c7-a141-66b7c0168a29 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.049199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2867e1b-31ef-40c7-a141-66b7c0168a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.057654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.070436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c2867e1b-31ef-40c7-a141-66b7c0168a29 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:05.071785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c2867e1b-31ef-40c7-a141-66b7c0168a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:05.097594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.162129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 241/484] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:05.194750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be25f6c-f453-4f6f-8efd-1287b6343916 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.207028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2867e1b-31ef-40c7-a141-66b7c0168a29 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.297014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd08c2-b821-446b-a104-81aea94edeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.297673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd08c2-b821-446b-a104-81aea94edeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.297953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd08c2-b821-446b-a104-81aea94edeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.298228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd08c2-b821-446b-a104-81aea94edeb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.309701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd08c2-b821-446b-a104-81aea94edeb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-55bd08c2-b821-446b-a104-81aea94edeb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:05.315718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:05.316627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:05.317464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 244/485] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:05.318518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302771-a34e-42d8-b4cb-a6503188d591 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.321235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302771-a34e-42d8-b4cb-a6503188d591 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.321235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27302771-a34e-42d8-b4cb-a6503188d591 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.324496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 245/486] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:05.325312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302771-a34e-42d8-b4cb-a6503188d591 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.375770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2867e1b-31ef-40c7-a141-66b7c0168a29 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.408721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.414215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1b0c5-e423-46c4-8e22-a5c1ab9eed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.415021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1b0c5-e423-46c4-8e22-a5c1ab9eed27 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.415303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1b0c5-e423-46c4-8e22-a5c1ab9eed27 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.415607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1b0c5-e423-46c4-8e22-a5c1ab9eed27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.426807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1b0c5-e423-46c4-8e22-a5c1ab9eed27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-a9e1b0c5-e423-46c4-8e22-a5c1ab9eed27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:05.432725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:05.434779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 242/487] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:05.436785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe57e8-0b9e-4c2a-8aaa-3781bbaf2a69 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.438496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe57e8-0b9e-4c2a-8aaa-3781bbaf2a69 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.438496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe57e8-0b9e-4c2a-8aaa-3781bbaf2a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.456700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3bfe57e8-0b9e-4c2a-8aaa-3781bbaf2a69 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:05.458166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3bfe57e8-0b9e-4c2a-8aaa-3781bbaf2a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:05.463753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27302771-a34e-42d8-b4cb-a6503188d591 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.472281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.506990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.507663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.507945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.508220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.509011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:05.509317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.511251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.524508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 246/488] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.525481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.544044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 243/489] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:05.544841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe57e8-0b9e-4c2a-8aaa-3781bbaf2a69 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.653804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8776308-ffb3-492e-b877-81f7c616f7c5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.666690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe57e8-0b9e-4c2a-8aaa-3781bbaf2a69 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.689359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.689982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.690262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.690536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.690782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.691502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:05.691632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.692953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.706646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 247/490] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.706646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.708916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.709575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.709855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.710128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.724604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.726107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:05.745142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:05.745988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:05.746808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:05.746808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:05.746808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:05.746808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:05.747605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.751024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 244/491] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:05.752196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.850444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c3f91-cd38-45aa-b637-3e83f6aff6a0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.866445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd52b5e-d1e9-454a-8f55-62bbb69df260 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.898384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.899454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.899953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.901081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.902893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.903681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.904120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.904461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.914912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:05.915902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.917496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:05.922158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:05.923018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:05.923790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 248/492] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:05.924483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:05.939474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:05.940552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:05.941566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:05.941566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:05.941566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:05.941566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:05.943542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 245/493] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:05.944492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.979384 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.028625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.037700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b210e6-e53a-401f-9029-6baa9b7c5a36 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.054033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c4225-2f57-42ec-b38b-63576f136eec None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.073157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750123-5540-443b-a861-28106729feaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.074006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750123-5540-443b-a861-28106729feaa None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.074287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750123-5540-443b-a861-28106729feaa None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.074564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750123-5540-443b-a861-28106729feaa None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.075264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750123-5540-443b-a861-28106729feaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:06.075568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750123-5540-443b-a861-28106729feaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.076979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750123-5540-443b-a861-28106729feaa None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.088539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 249/494] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.089322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750123-5540-443b-a861-28106729feaa None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.092843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f81a43-1578-4727-92dd-eb2f0fa3d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.093466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f81a43-1578-4727-92dd-eb2f0fa3d628 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.093745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f81a43-1578-4727-92dd-eb2f0fa3d628 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.094022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f81a43-1578-4727-92dd-eb2f0fa3d628 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.094752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f81a43-1578-4727-92dd-eb2f0fa3d628 None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:06.095063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f81a43-1578-4727-92dd-eb2f0fa3d628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.096449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f81a43-1578-4727-92dd-eb2f0fa3d628 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.108822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 246/495] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.108822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb0fe0-ceb9-481b-ab83-45b6938ea15f None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.108822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb0fe0-ceb9-481b-ab83-45b6938ea15f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.108822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb0fe0-ceb9-481b-ab83-45b6938ea15f None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.108822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 247/496] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:06.108822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9be5-8c09-4ae4-a3a8-ccc88a931f85 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.108822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9be5-8c09-4ae4-a3a8-ccc88a931f85 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.109306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9be5-8c09-4ae4-a3a8-ccc88a931f85 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.111716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 248/497] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:06.111716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9be5-8c09-4ae4-a3a8-ccc88a931f85 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.193777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750123-5540-443b-a861-28106729feaa None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.202326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.214906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9be5-8c09-4ae4-a3a8-ccc88a931f85 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.227634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.228330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.228731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.229069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.229759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:06.230061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.231425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.247602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 250/498] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.247602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.248938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97b700-21fa-486b-9a81-d985f975f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.249573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b700-21fa-486b-9a81-d985f975f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.249858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b700-21fa-486b-9a81-d985f975f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.250132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b700-21fa-486b-9a81-d985f975f4a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.255601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.257303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97b700-21fa-486b-9a81-d985f975f4a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.258738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97b700-21fa-486b-9a81-d985f975f4a4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-6f97b700-21fa-486b-9a81-d985f975f4a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:06.271859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:06.273031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 249/499] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:06.274200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278ee81-e663-4b5a-a341-bc8812b9fc2c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.274870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278ee81-e663-4b5a-a341-bc8812b9fc2c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.274870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278ee81-e663-4b5a-a341-bc8812b9fc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.292242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3278ee81-e663-4b5a-a341-bc8812b9fc2c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:06.293484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3278ee81-e663-4b5a-a341-bc8812b9fc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:06.352069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673a530-ca6c-4cfa-8a6a-e81ccf26e8d4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.361962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 250/500] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:06.363286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e295-4245-49e1-8648-c963f09455a7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.363578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e295-4245-49e1-8648-c963f09455a7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.363854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e295-4245-49e1-8648-c963f09455a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.381406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-bbb0e295-4245-49e1-8648-c963f09455a7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:06.382591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bbb0e295-4245-49e1-8648-c963f09455a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:06.386660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.387299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.387579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.387853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.398343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.399794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:06.415184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:06.416602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:06.417449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:06.417449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:06.417449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:06.417449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:06.417449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 251/501] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:06.417705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.457935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 251/502] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:06.459101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0e295-4245-49e1-8648-c963f09455a7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.472974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.486627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.523974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772bf8c2-52d3-4223-a3e2-e448e330018b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.558212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.558885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.559145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.559413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.560543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0e295-4245-49e1-8648-c963f09455a7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.566778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.568205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:06.580637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:06.581683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 252/503] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:06.582576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.597317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.597936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.598222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.598499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.599301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:06.599607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.601063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.638606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 252/504] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.639405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.682379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d93e5-540d-4128-87d8-fb1c3b795806 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.705493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.709980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.717235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.717870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.718150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.718430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.729641 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:06.735050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:06.736055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:06.737011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 253/505] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:06.737850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.749899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51363981-88c6-4cb7-b8f4-014aaf6697ac None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.785189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.785813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.786095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.786370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.787071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:06.787379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.788801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.806846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 253/506] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.807651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.842961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c28e7-9bd2-49ec-9377-2fce932061ec None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.876874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.877507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.877790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.878067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.888920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.890357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.904435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb48253-93ca-4267-a7ab-d7c0b4fa1c60 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:06.904885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:06.905814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:06.906741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:06.906741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:06.906741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:06.906741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:06.906741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 254/507] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:06.906998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.933066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.939980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.941250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.941864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.942142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.942414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.943208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:06.943514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.944981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.961465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 254/508] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.962190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.010099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6d811-0ca6-4abd-a859-62a5bfb66673 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.048045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.048803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.049088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.049355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.064584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a183a4-44a2-474b-9f76-43c1ed3222e5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.098855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.099470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.099755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.100035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.100789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:07.101101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.102498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.110357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 255/509] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:07.111135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.121350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.122798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.156342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.156893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.207309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22df13-3f29-4f05-98a0-be1f6a8b74a1 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.222023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 255/510] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:07.222812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.263474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.263474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.263474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.263934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.276039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.277555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:07.289785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:07.290732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:07.291595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 256/511] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:07.292735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.359075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e5898-1720-449e-a8d5-7ad4b551b584 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.395648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.396656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.397148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.397601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.398291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:07.398593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.400647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.402395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.407037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dff2ca-0677-4987-8085-5717344ea41c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.412914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 256/512] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:07.413749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.414870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.446382 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.447682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.447965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.448239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.521044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacc0cf-88e8-4ff3-84b3-f3769366d081 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.525387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.526845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.555447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.556632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.556928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.557203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.567856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.569349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:07.584321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:07.585379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:07.586413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:07.586413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:07.586413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:07.586413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:07.586413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 257/513] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:07.586413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.630535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 257/514] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:07.631572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.636110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.640677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.687227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9e822-7f01-4c87-9b84-a67a781b885b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.720888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.721605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.721965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.722312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.732688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.734141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.734862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6004e3e-34cd-454e-a4c9-a26a608c60d1 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:07.748844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:07.749805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:07.750815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:07.750815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:07.750815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:07.750815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:07.750815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 258/515] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:07.750815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.773314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.773842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.774120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.774393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.775221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:07.775528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.776992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.793654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 258/516] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:07.794560 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.847357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.851388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441334a-fc68-4ad3-8956-719ea326abcc None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.854840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.886008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.886624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.887002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.887281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.887957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.889430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.891562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827a6326-89de-482b-b32d-227d36b7eb98 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.925226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.925852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.926131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.926404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.927094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:07.927393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.928784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.939508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 259/517] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:07.940354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.995725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 259/518] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:07 2022] POST /identity/v3/users => generated 247 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:07.996593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.044098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38297cb-f1d7-48e8-86a3-c02d7efa5b92 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.075609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.088484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.089409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.089829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.090241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.101549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.103126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:08.120460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:08.121460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:08.122298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:08.122298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:08.122298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:08.122298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:08.122298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 260/519] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:08.122560 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.137099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbdb6d-059e-488f-80a3-f7c7bb0ae12a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.173814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.174446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.174721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.174995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.175684 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:08.175986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.177473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.190349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 260/520] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:08.191189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.209463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.243391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5947889-8bf1-4c72-9a6d-21370b886ba4 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.279279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.280096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.280509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.280889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.297426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9cd03-793e-4512-b6fb-fdd6610b7867 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.329163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.339048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.342448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.342448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.342448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.349712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.351278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.351788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.353271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:08.363853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:08.364986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:08.366037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 261/521] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:08.366811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.450909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.459231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 261/522] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:08.460326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.484596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd1eb-0465-4151-afde-2e61d3f5e1eb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.523155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.523785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.524063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.524339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.525090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:08.525395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.526768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.538723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 262/523] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:08.539599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.546456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.569409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f441b-a820-42ff-bb27-c260bfd48bbc None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.607319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.608100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.608536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.608919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.615734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.617217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:08.629947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:08.630954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:08.632057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:08.635859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 262/524] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:08.636729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.653247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03156bd0-e905-4d51-9c7b-12c7eeb7e1ad None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.695403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.699253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.699253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.699253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.699722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:08.700020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.703973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.706320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.746253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ae918-a27f-4c07-b039-521d37845634 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.757079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 263/525] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:08.757973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.781415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.782063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.782343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.782632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.785449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.793576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.795058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:08.810605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:08.811791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:08.812919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:08.812919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:08.812919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:08.812919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:08.816513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 263/526] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:08.817380 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.868072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd2358-3fcd-4ed1-90ab-d9f9b4e2eb58 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.903349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.904096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.904307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.904607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.911634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.913100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.918091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4ae09-5aec-49a7-a906-899e6b4aa139 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:08.928984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:08.929820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:08.930647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:08.935480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 264/527] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:08.936094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.954058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.954694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.954961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.955238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.955992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:08.956304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.957761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.981614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 264/528] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:08.981614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.998736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.034257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.048284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741869-24c4-4fb4-91e7-7c62b317fe7c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.083778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.084206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b49406-61d1-49ea-bc42-c77e052b5d94 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.084435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.084704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.084980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.123449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.124000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.124284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.125002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.125716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:09.126026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.127774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.146606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 265/529] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.146606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.165006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.167036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.220917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.259510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b72193-6291-4439-8cb5-c141a05a6893 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.269486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.274422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 265/530] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:09.274422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.300066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.300710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.300991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.301292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.302077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:09.302388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.303812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.320613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 266/531] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.320613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.392038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea0441-01b0-4755-b70c-44a167bcc5e9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.429537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.430197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.430419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.430738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.432014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=784880e276bf4d64ae785609a463403a) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:09.432320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.432555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb67ec-234f-46d2-aa45-3dfa8539d7b2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.433773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.444731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 266/532] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/role_assignments?user.id=784880e276bf4d64ae785609a463403a&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.444731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.468813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.492914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.495062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.495349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.495631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.506758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.509207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:09.529463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:09.530521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:09.531546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:09.535027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 267/533] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:09.535027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.576160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d1d2a-6132-4442-ace2-35986212dbe5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.577769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.621281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.621942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.622222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.622494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.636030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.637531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:09.653503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:09.654359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:09.655199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:09.655199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:09.655199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:09.655199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:09.659182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 267/534] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:09.660089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.683775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea98b1-ce09-4e81-9df2-3a6407553559 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.727389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.728785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.728968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.729197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.730059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=813427e9e85a4b48a550a290211f39b5) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:09.730307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.731771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.744427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 268/535] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/role_assignments?user.id=813427e9e85a4b48a550a290211f39b5&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.744427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.746876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.818732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235247a-f05f-413d-8b06-4aef97c58ba5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.896947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.897589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.897923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.898206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.898988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:09.899344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.901262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.926386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.931646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 268/536] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.938737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.946108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c30d-3964-46ae-a708-f3c8b4d8c8be None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.994185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.995628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.996149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.996696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.997602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:09.998126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.000095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.024307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 269/537] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:10.029738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.100808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb59289-9f93-4fa3-8478-7b5856872772 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.131216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.140308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1eff9-5e71-40c7-b25b-a762c2834258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.158619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1eff9-5e71-40c7-b25b-a762c2834258 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.158619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1eff9-5e71-40c7-b25b-a762c2834258 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.158619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1eff9-5e71-40c7-b25b-a762c2834258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.158619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1eff9-5e71-40c7-b25b-a762c2834258 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=78776208cde94134b478ceee6cc70ec8) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:10.158619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1eff9-5e71-40c7-b25b-a762c2834258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.158619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1eff9-5e71-40c7-b25b-a762c2834258 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.159063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 269/538] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/role_assignments?user.id=78776208cde94134b478ceee6cc70ec8&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:10.159130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f575e7-8c9b-4b0c-a0bb-f86090212ff9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.160064 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48c7f1-c407-4fb7-a1da-1d8266ebd8a8 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.160558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48c7f1-c407-4fb7-a1da-1d8266ebd8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.161028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48c7f1-c407-4fb7-a1da-1d8266ebd8a8 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.163087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 270/539] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:10 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:10.170499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120cbb0-53a1-4545-b8ce-55268b387ae2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.171121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120cbb0-53a1-4545-b8ce-55268b387ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.171579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120cbb0-53a1-4545-b8ce-55268b387ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.202303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.202895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.203197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.203446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.205006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a120cbb0-53a1-4545-b8ce-55268b387ae2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:10.206242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a120cbb0-53a1-4545-b8ce-55268b387ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:10.217672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.219152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:10.240786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:10.241690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:10.242930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 270/540] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:10.251337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.278902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.378891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515ef78-1455-415b-acf6-058169baa19b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.398040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 271/541] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:10.427930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.428583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.428867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.429144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.429903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:10.430212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.432175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.449097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 271/542] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:10.457465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.457593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120cbb0-53a1-4545-b8ce-55268b387ae2 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.528990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.559981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b699-70d7-4c6a-a772-a1e8aec1ca66 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.594729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.595350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.595630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.595902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.596734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=54aedfbb64c54c589003b335fde2a17b) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:10.597045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.598482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.604555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 272/543] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/role_assignments?user.id=54aedfbb64c54c589003b335fde2a17b&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:10.676617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120cbb0-53a1-4545-b8ce-55268b387ae2 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.704778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.724511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.724511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.724511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.724511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.742098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:10.748260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:10.749215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:10.749996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 272/544] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:10.755565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.877078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de0265-e549-414e-baa1-b9fcd4bc8b2e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.931539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.931819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.932090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.932839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:10.933143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.934525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.945337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.945720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 273/545] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:10.950759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.088903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd38df-bb7e-4aa8-8df8-f9f71b6015f7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.137107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.137770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.138050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.138323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.152528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.154112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:11.172665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:11.174628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:11.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:11.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:11.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:11.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:11.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 273/546] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:11.179862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.310272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af2c80-c290-4b85-b3af-389f4fe7d30d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.363472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.364111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.364371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.364672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.365373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:11.365676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.367043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.386748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 274/547] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:11.391845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.436478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:11.511841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80f63e-c930-4ea1-8fd2-08fbc2f9605e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.554422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.555133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.555443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.555958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.563577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.565043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:11.576811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:11.577891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:11.579505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 274/548] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:11.584014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.661153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:11.710561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473b609-9d8f-4bdf-9b94-07ec9fc89252 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.745730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22daf7-32d7-40bb-8935-4ca326eed598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.746348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22daf7-32d7-40bb-8935-4ca326eed598 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.746627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22daf7-32d7-40bb-8935-4ca326eed598 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.746899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22daf7-32d7-40bb-8935-4ca326eed598 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.747702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22daf7-32d7-40bb-8935-4ca326eed598 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:11.748006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22daf7-32d7-40bb-8935-4ca326eed598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.749464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22daf7-32d7-40bb-8935-4ca326eed598 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.776171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 275/549] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:11.781770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.909266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:11.916124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0b220-2610-41cd-8ed3-9e4ad1f6a021 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.916253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce0fce-0386-414a-9b01-83199eaa3c86 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.916813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0b220-2610-41cd-8ed3-9e4ad1f6a021 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.917308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0b220-2610-41cd-8ed3-9e4ad1f6a021 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.919038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 276/550] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:11.924481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d471b-4436-40af-81e2-6b77fddc234e None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.924964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d471b-4436-40af-81e2-6b77fddc234e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.925410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d471b-4436-40af-81e2-6b77fddc234e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.948901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-358d471b-4436-40af-81e2-6b77fddc234e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:11.951475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-358d471b-4436-40af-81e2-6b77fddc234e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:11.973442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed08df9-80fd-4e01-8285-d201e2daec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.975104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed08df9-80fd-4e01-8285-d201e2daec94 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.975776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed08df9-80fd-4e01-8285-d201e2daec94 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.977739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed08df9-80fd-4e01-8285-d201e2daec94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.081378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 277/551] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:12.093581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed08df9-80fd-4e01-8285-d201e2daec94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.095306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed08df9-80fd-4e01-8285-d201e2daec94 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.138621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d471b-4436-40af-81e2-6b77fddc234e None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.216490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.243225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.253706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 275/552] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:12.255101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba640a4f-e51c-4e13-9d35-23407adf88c3 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.255394 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba640a4f-e51c-4e13-9d35-23407adf88c3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.255678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba640a4f-e51c-4e13-9d35-23407adf88c3 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.257308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 276/553] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:12.259205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba640a4f-e51c-4e13-9d35-23407adf88c3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.290610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d471b-4436-40af-81e2-6b77fddc234e None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.326199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8360ae6-d240-42cd-a5bd-a1425867b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.326849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8360ae6-d240-42cd-a5bd-a1425867b552 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.327169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8360ae6-d240-42cd-a5bd-a1425867b552 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.327453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8360ae6-d240-42cd-a5bd-a1425867b552 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.341491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8360ae6-d240-42cd-a5bd-a1425867b552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-e8360ae6-d240-42cd-a5bd-a1425867b552 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:12.348538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:12.349458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:12.350663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:12.355110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 278/554] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:12.356437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151503-8b11-45dd-8112-ae2b254ed17d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.356756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151503-8b11-45dd-8112-ae2b254ed17d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.357066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151503-8b11-45dd-8112-ae2b254ed17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.375609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fd151503-8b11-45dd-8112-ae2b254ed17d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:12.376772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fd151503-8b11-45dd-8112-ae2b254ed17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:12.379831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba640a4f-e51c-4e13-9d35-23407adf88c3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.416185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.417216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.417723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.418225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.429637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.431967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.447601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 279/555] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:12.448479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:12.449317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:12.450170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:12.450170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:12.450170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:12.450170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:12.450383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151503-8b11-45dd-8112-ae2b254ed17d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.452332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 277/556] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:12.457301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.520820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.523166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.559526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151503-8b11-45dd-8112-ae2b254ed17d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.590381 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeab63-637e-4264-86bc-2a5f63837e47 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.605440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.606851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.607639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.608064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.609341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:12.609816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.612033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.629095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 280/557] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:12.630705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.640174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.641053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.641395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.642491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.643088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:12.643471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.645525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.662226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 278/558] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:12.663640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.740825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.745122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8dee2-336a-409a-b559-577eaff8d57a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.782151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97cb34-9091-4be0-a9ee-e6c6fbafe510 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.784293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75ff41-ca26-4af8-8313-02b38bea007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.785276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75ff41-ca26-4af8-8313-02b38bea007a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.785276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75ff41-ca26-4af8-8313-02b38bea007a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.785497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75ff41-ca26-4af8-8313-02b38bea007a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.791495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.797105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75ff41-ca26-4af8-8313-02b38bea007a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-9b75ff41-ca26-4af8-8313-02b38bea007a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:12.802143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:12.809157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 281/559] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:12.810001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42ad3-652f-4152-9ad6-8d22c9407901 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.810877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42ad3-652f-4152-9ad6-8d22c9407901 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.810877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad42ad3-652f-4152-9ad6-8d22c9407901 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.810877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 282/560] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:12.810877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad42ad3-652f-4152-9ad6-8d22c9407901 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.828156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.829403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.829960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.830595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.841817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.843257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:12.858428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:12.859535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:12.860851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:12.860851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:12.860851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:12.860851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:12.864272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 279/561] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:12.865118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.920834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad42ad3-652f-4152-9ad6-8d22c9407901 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.955433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ec974-a104-4a2e-9f8e-cac1bf3df362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.956061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ec974-a104-4a2e-9f8e-cac1bf3df362 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.956338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ec974-a104-4a2e-9f8e-cac1bf3df362 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.956642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ec974-a104-4a2e-9f8e-cac1bf3df362 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.964367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ec974-a104-4a2e-9f8e-cac1bf3df362 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.965828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ec974-a104-4a2e-9f8e-cac1bf3df362 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.969759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffbef0-81e3-4f4d-bc75-6b9fc4f0befe None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-c53ec974-a104-4a2e-9f8e-cac1bf3df362 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:12.978262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:12.979107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:12.979990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 283/562] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:12.981099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43071fd4-6c4e-48a8-81cd-7c0b99d6bcf1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.981099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43071fd4-6c4e-48a8-81cd-7c0b99d6bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.981099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43071fd4-6c4e-48a8-81cd-7c0b99d6bcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.999762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-43071fd4-6c4e-48a8-81cd-7c0b99d6bcf1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:13.000982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-43071fd4-6c4e-48a8-81cd-7c0b99d6bcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:13.009835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.010670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.010798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.011174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.011980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.012420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.013861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.014744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.022629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 280/563] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.023461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.047520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.074143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 284/564] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:13.074981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43071fd4-6c4e-48a8-81cd-7c0b99d6bcf1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.124253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac15a5-dd12-4d0d-be4b-18de574b9539 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.157304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.157923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.158205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.158477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.159198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.159499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.160907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.172719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 281/565] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.173499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.180916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43071fd4-6c4e-48a8-81cd-7c0b99d6bcf1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.215419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612890df-05db-47a5-999e-13d9b99a43d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.216038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612890df-05db-47a5-999e-13d9b99a43d0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.216321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612890df-05db-47a5-999e-13d9b99a43d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.216627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612890df-05db-47a5-999e-13d9b99a43d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.217389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612890df-05db-47a5-999e-13d9b99a43d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.217702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612890df-05db-47a5-999e-13d9b99a43d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.219132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612890df-05db-47a5-999e-13d9b99a43d0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.230581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 285/566] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.231728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccd821-827e-4b08-8323-b954779f3e28 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.232091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccd821-827e-4b08-8323-b954779f3e28 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.232373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccd821-827e-4b08-8323-b954779f3e28 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.233613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 286/567] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:13.234300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccd821-827e-4b08-8323-b954779f3e28 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.245558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.257688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.277311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92297aa5-2bd7-4690-b0d4-f3ab24c15643 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.310724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.311297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.311579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.311851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.322237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.323666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.334933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccd821-827e-4b08-8323-b954779f3e28 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:13.338736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:13.339595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:13.345211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:13.345211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:13.345211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:13.345211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:13.345211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 282/568] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:13.345211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.370819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.371451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.371725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.372000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.381857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:13.387702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:13.388819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:13.389636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:13.392656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 287/569] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:13.393325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.444001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7671d-d801-4594-bafd-d6eb60674152 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.458898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.472831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.477550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12f303-c2ae-4d01-b66c-c97403d50746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.478180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12f303-c2ae-4d01-b66c-c97403d50746 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.478460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12f303-c2ae-4d01-b66c-c97403d50746 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.478731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12f303-c2ae-4d01-b66c-c97403d50746 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.486231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12f303-c2ae-4d01-b66c-c97403d50746 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.487655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12f303-c2ae-4d01-b66c-c97403d50746 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.490093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570f21b-3f53-4ccd-9787-99998d2c10cb None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0e12f303-c2ae-4d01-b66c-c97403d50746 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:13.499295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:13.502337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 283/570] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cceb0dd-e27b-4617-8d2f-722626b4a912 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.503449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cceb0dd-e27b-4617-8d2f-722626b4a912 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.504285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cceb0dd-e27b-4617-8d2f-722626b4a912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.520429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7cceb0dd-e27b-4617-8d2f-722626b4a912 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:13.521899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7cceb0dd-e27b-4617-8d2f-722626b4a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:13.528680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.529299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.529588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.529865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.530659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=dcebfbf0b3084929bc33887467695b00) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.530971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.532416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.537707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 288/571] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/role_assignments?user.id=dcebfbf0b3084929bc33887467695b00&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.538470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.608937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 284/572] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:13.610105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cceb0dd-e27b-4617-8d2f-722626b4a912 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.667853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8bce6-4cb1-4cf3-8fd3-8a3d12101f51 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.704712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.709801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.710423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.710706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.711015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.711718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.712023 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.713521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.726200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 289/573] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.726955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.736030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cceb0dd-e27b-4617-8d2f-722626b4a912 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.761446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.774316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.774950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.775226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.775502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.776261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.776571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.777943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.786582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 285/574] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.787378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.839118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8413e-a0ac-4e8e-99e2-ea9e60f32bf7 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.875120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.876098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.876607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.877095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.878121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.878534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.880075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.895778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc80e2-b286-4781-8f27-d927be947df4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.898028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 290/575] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.899596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.934825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.935477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.935754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.936031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.947118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:13.952192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:13.953120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:13.954179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 286/576] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:13.955166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.004546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.010982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41067e-11c9-4226-ae52-305953ed09b8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.048594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.049232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.049514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.049787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.057889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.059337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.062537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9de90-5290-4377-be57-060bca38c7af None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:14.071609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.072582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 291/577] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.073556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.102848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.103758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.104038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.104301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.114802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.116258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:14.133695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.134598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.135486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:14.135486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:14.135486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.135486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:14.135486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 287/578] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.135771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.184507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed755109-4aa1-4669-9fab-59119e0888a3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.196346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.222362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.223033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.223270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.223545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.234566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.236011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.245643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:14.251485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.252344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.253239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:14.253239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:14.253239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.253239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:14.253452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e0744-49e8-4416-b61e-5b4dd77f8679 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.253530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 292/579] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.254030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.293664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.294529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.294808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.295085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.295794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.296095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.297467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.306354 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 288/580] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:14.307144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.369184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4765f5-8c2f-4ab9-8712-d3a7fdf6ed92 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.409156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.409699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.409983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.410261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.411097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.411407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.412900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.419774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9139600-3fa8-45ef-af48-554a48126578 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.435944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 293/581] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:14.435944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.455899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.459522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.460238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.460622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.461424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.462115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.462422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.463790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.477237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 289/582] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:14.481917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.500992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.540825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb97ba-0105-4b1e-a2d9-d7cf93782a9c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.577609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45401d-267f-496c-987a-630456794f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.578335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45401d-267f-496c-987a-630456794f1d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.578616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45401d-267f-496c-987a-630456794f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.578890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45401d-267f-496c-987a-630456794f1d None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.579587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45401d-267f-496c-987a-630456794f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.579895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45401d-267f-496c-987a-630456794f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.581522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45401d-267f-496c-987a-630456794f1d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.583757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461ed30-580a-43a6-bcb6-49a1201d878f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.595458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 294/583] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:14.596674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45401d-267f-496c-987a-630456794f1d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.623553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.624192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.624504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.624782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.637739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.639242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:14.656205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.657141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.658030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:14.658030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:14.658030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.658030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:14.659590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 290/584] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.660545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.685541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.705592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45401d-267f-496c-987a-630456794f1d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.739944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.741974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.742633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.742918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.743198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.756612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.758651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.773051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22566c-fd05-4743-b39e-ca60b9a862b6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:14.773219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.774606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.775917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:14.775917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:14.775917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:14.775917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:14.775917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 295/585] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.776188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.821851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.822383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.822667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.822941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.832822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.834244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:14.845988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.846874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:14.847782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 291/586] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.848350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.884259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0fded-ab74-4697-a010-cce0e6741d91 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.916050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.921449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.922269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.922552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.922827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.930543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.932004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:14.943678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.944787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:14.946261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 296/587] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.946734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.961717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fb7e0-92d7-4868-9521-fbeb30a6ed2a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.990659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.997310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.997932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.998217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.998489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.999184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.999489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.001008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.018874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 292/588] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:15.018874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.056111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb035e-c029-44ee-bde9-16605ab40e54 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.095340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.096244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.096719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.097195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.098064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:15.098573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.100629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.118558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 297/589] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:15.119598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.130158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925c072-4495-48d6-8f8f-a26a442e2305 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.166514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.167379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.167662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.167932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.168778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:15.169337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.170756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.178229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.193664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 293/590] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:15.193664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.232993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3348e2-a931-420c-b353-848d35e4daab None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.244008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.272906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.273553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.273837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.274112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.274876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:15.275186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.276635 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.288264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 298/591] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:15.289131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.301506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911bee8-7dd8-4aca-a8a3-29b1242ae9dd None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.343340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.343979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.344253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.344556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.352086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.353553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:15.365653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:15.366559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 294/592] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:15.368046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.407650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc132ed3-cf6a-49b6-81f6-05265e63b376 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.424747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.445725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.446361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.446694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.446967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.454200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.455825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:15.468701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:15.469570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:15.470441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 299/593] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:15.470916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.478442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ac10e-924e-4db4-ac9d-a59ab55d6b8f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.487857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.512957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.513691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.514036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.514377 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.525255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.526765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:15.541742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:15.542861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:15.543884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:15.543884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:15.543884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:15.543884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:15.543884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 295/594] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:15.543884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.576332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255398ed-ea13-414a-9681-e0f6df12364f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.608923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92186c5-619a-494e-a78e-85f113a5c7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.609584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92186c5-619a-494e-a78e-85f113a5c7b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.609869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92186c5-619a-494e-a78e-85f113a5c7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.610148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92186c5-619a-494e-a78e-85f113a5c7b1 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/784880e276bf4d64ae785609a463403a/roles/4981f801f6b14138984b3ae833a5797a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.624559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92186c5-619a-494e-a78e-85f113a5c7b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=784880e276bf4d64ae785609a463403a, role_id=4981f801f6b14138984b3ae833a5797a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.626006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92186c5-619a-494e-a78e-85f113a5c7b1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.641549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308221c-d1fe-4eb5-b990-1355ab0e8bf2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.674189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.674874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.675110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.675383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.676172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:15.676512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.677910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.695061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 296/595] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:15.695877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.707926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 300/596] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:15 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/784880e276bf4d64ae785609a463403a/roles/4981f801f6b14138984b3ae833a5797a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:15.709209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177046b4-2a3d-4352-ba2e-081848a88315 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.709502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177046b4-2a3d-4352-ba2e-081848a88315 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.709780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177046b4-2a3d-4352-ba2e-081848a88315 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.711704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 301/597] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:15.714772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.716881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177046b4-2a3d-4352-ba2e-081848a88315 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.746775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.816181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22afb-84bf-4298-89c2-b44a44e4be5e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.850281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177046b4-2a3d-4352-ba2e-081848a88315 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.853636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.854269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.854550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.854827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.855591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:15.855902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.857370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.872191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 297/598] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:15.873197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.894254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e7a4c-6d62-456f-9a4f-8de86c2e54ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.894896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e7a4c-6d62-456f-9a4f-8de86c2e54ec None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.895184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e7a4c-6d62-456f-9a4f-8de86c2e54ec None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.895464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e7a4c-6d62-456f-9a4f-8de86c2e54ec None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.896181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e7a4c-6d62-456f-9a4f-8de86c2e54ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:15.896592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e7a4c-6d62-456f-9a4f-8de86c2e54ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.898898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e7a4c-6d62-456f-9a4f-8de86c2e54ec None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.910638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 302/599] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:15.911835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340bd9b-38bb-4a15-bae1-86a47a6cbb4a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.912121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340bd9b-38bb-4a15-bae1-86a47a6cbb4a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.912431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340bd9b-38bb-4a15-bae1-86a47a6cbb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.930845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-0340bd9b-38bb-4a15-bae1-86a47a6cbb4a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:15.931968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0340bd9b-38bb-4a15-bae1-86a47a6cbb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:15.971924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.997020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8cefd-5925-48a3-b15b-2d1375665f9b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.000941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 303/600] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:16.001828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340bd9b-38bb-4a15-bae1-86a47a6cbb4a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.016707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:16.052242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.053031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.053316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.053593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.069078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:16.071183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:16.086292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:16.087361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:16.088470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:16.088470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:16.088470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:16.088470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:16.092909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 298/601] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:16.093608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.110868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340bd9b-38bb-4a15-bae1-86a47a6cbb4a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.150076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.150934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.151322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.151757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/813427e9e85a4b48a550a290211f39b5/roles/4981f801f6b14138984b3ae833a5797a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.166564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=813427e9e85a4b48a550a290211f39b5, role_id=4981f801f6b14138984b3ae833a5797a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:16.169662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:16.204698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9692b0-e8b4-45ad-9f7a-64d944b104b9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.232490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:16.238295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.239069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.248703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 304/602] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:16 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/813427e9e85a4b48a550a290211f39b5/roles/4981f801f6b14138984b3ae833a5797a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:16.249250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.249250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.249451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.258094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:16.259791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:16.272494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:16.274256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:16.275327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:16.278948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 299/603] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:16.279947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.327896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:16.393743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f427a2c-6964-4f8c-b66d-5698fcaf769b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.408481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fcca4-3e8c-479c-8925-ddddfe9e3d43 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.444859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.446224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.446673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.447130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.452510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.453326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.453608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.453880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.454572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:16.454923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:16.456222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:16.459017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:16.464529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:16.465659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:16.466624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 305/604] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:16.467438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.478317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 300/605] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:16.479565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.565016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:16.648596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e7986-66f2-4332-bb97-c0664905ad98 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.649853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6baa0-c0bd-4684-8aa1-2ccc8ec8a5b6 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.689987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:16.708608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.710071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.710250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.711065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.711242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.711583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.712056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.712574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.713423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:16.714065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:16.716892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:16.725174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:16.728105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:16.747676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 306/606] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:16.747676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:16.758019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:16.759696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:16.761482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:16.772491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 301/607] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:16.773945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.891299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340c58-4a8a-49ed-806f-292d383029f2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.933692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f12f0-99aa-456b-9860-26ff9cfae65d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:16.936983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.937596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.937870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.938132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.938846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:16.939144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:16.940526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:16.949394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 307/608] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:16.950200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:16.951545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:16.982491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:16.983419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:16.983697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:16.984071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/78776208cde94134b478ceee6cc70ec8/roles/4981f801f6b14138984b3ae833a5797a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:16.999388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:17.001845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=78776208cde94134b478ceee6cc70ec8, role_id=4981f801f6b14138984b3ae833a5797a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:17.003416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:17.056175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689d6e8-bfb8-492f-9396-6de1e1b15fac None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:17.089499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:17.090111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.090387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.090704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.091453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:17.091758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:17.093232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:17.110503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 308/609] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:17.110503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:17.123343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 302/610] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:16 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/78776208cde94134b478ceee6cc70ec8/roles/4981f801f6b14138984b3ae833a5797a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:17.133935 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:17.271757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55617c-8d8e-4ff2-9c7d-48e19d538e16 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:17.308606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43658635-2659-4862-9ce4-b18fac24b131 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:17.314096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:17.317439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:17.318387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.318913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.319198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/54aedfbb64c54c589003b335fde2a17b/roles/4981f801f6b14138984b3ae833a5797a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.340157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=54aedfbb64c54c589003b335fde2a17b, role_id=4981f801f6b14138984b3ae833a5797a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:17.343079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:17.367090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:17.367738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.368024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.368304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.379296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:17.380802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:17.397158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:17.398599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:17.399482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:17.399482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:17.399482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:17.399482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:17.399482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 309/611] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:17.405700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:17.427859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 303/612] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:17 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/54aedfbb64c54c589003b335fde2a17b/roles/4981f801f6b14138984b3ae833a5797a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:17.560612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df0c25-3acf-4875-bc1a-0fa98d188a46 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:17.610822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:17.611819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.612328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.612840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.613739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:17.614251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:17.615867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:17.630674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 310/613] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:17.637291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:17.660109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:17.782192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702437e-92a8-482b-a9ca-acf38d5d8ea7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:17.828016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd834e-091e-4cce-82f9-caeadcb0da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:17.828610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd834e-091e-4cce-82f9-caeadcb0da80 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.828887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd834e-091e-4cce-82f9-caeadcb0da80 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.829158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd834e-091e-4cce-82f9-caeadcb0da80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.837385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd834e-091e-4cce-82f9-caeadcb0da80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:17.838825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd834e-091e-4cce-82f9-caeadcb0da80 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-9ffd834e-091e-4cce-82f9-caeadcb0da80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:17.856710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:17.857679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:17.858585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:17.861626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 304/614] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:17.867529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:17.906672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.001924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1b10-fbf4-4edb-95ea-e2b235994186 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.002206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1b10-fbf4-4edb-95ea-e2b235994186 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.002481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad1b10-fbf4-4edb-95ea-e2b235994186 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.004087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 305/615] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:18.021825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478af0c-ce2d-4d6f-92a0-5416f8ce84b3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.022254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478af0c-ce2d-4d6f-92a0-5416f8ce84b3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.022673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478af0c-ce2d-4d6f-92a0-5416f8ce84b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.038019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607faa8-4351-4a6d-bec8-d69cb977bb70 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.060285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e478af0c-ce2d-4d6f-92a0-5416f8ce84b3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:18.061670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e478af0c-ce2d-4d6f-92a0-5416f8ce84b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:18.074214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.074846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.075130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.075409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.076223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:18.076556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.077995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.095821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 311/616] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:18.101187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.157544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 306/617] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:18.158839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.199306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478af0c-ce2d-4d6f-92a0-5416f8ce84b3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.236736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.284997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ead74-9c46-48b0-8401-7e441e930281 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.354543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478af0c-ce2d-4d6f-92a0-5416f8ce84b3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.391163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.391864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.392147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.392444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.397105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.398324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.398612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.398991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.403712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:18.409233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:18.410145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:18.411114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 307/618] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:18.420926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.421316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.422397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:18.446365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:18.448581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:18.450589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:18.450589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:18.450589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.450589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:18.450589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 312/619] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:18.456952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.516308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.577870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fb226-97b7-46c2-9f33-6837feb744b6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.614648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6eea7-6836-48a0-92fb-1caf5feecc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.615190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6eea7-6836-48a0-92fb-1caf5feecc9d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.615471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6eea7-6836-48a0-92fb-1caf5feecc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.615746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6eea7-6836-48a0-92fb-1caf5feecc9d None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.616528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6eea7-6836-48a0-92fb-1caf5feecc9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:18.616833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6eea7-6836-48a0-92fb-1caf5feecc9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.617967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bf6c0-24e2-43ed-88a0-157473ce1cfa None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.618207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6eea7-6836-48a0-92fb-1caf5feecc9d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.632460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 308/620] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:18.632460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9445-587c-443f-b185-c2b0c990a4fd None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.632460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9445-587c-443f-b185-c2b0c990a4fd None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.632460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9445-587c-443f-b185-c2b0c990a4fd None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.636471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 309/621] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:18.637269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9445-587c-443f-b185-c2b0c990a4fd None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.658791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.670217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41020e17-4480-4249-bcef-a5b5e1b7aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.670850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41020e17-4480-4249-bcef-a5b5e1b7aebd None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.671142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41020e17-4480-4249-bcef-a5b5e1b7aebd None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.671469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41020e17-4480-4249-bcef-a5b5e1b7aebd None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.672565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41020e17-4480-4249-bcef-a5b5e1b7aebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:18.672875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41020e17-4480-4249-bcef-a5b5e1b7aebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.675053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41020e17-4480-4249-bcef-a5b5e1b7aebd None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.696470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 313/622] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:18.699766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274d301-bccb-470d-9a07-3210532d7f1c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.700057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274d301-bccb-470d-9a07-3210532d7f1c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.700331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274d301-bccb-470d-9a07-3210532d7f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.725812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3274d301-bccb-470d-9a07-3210532d7f1c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:18.727810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3274d301-bccb-470d-9a07-3210532d7f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:18.777341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9445-587c-443f-b185-c2b0c990a4fd None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.809400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.821861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.830482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.830482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.830482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.830750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 314/623] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:18.830750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274d301-bccb-470d-9a07-3210532d7f1c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.841920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.843509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:18.874831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:18.875732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:18.876639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:18.876639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:18.876639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.876639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:18.876639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 310/624] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:18.877962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.968169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274d301-bccb-470d-9a07-3210532d7f1c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.984217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d98221-136c-4173-aa4e-6d55e6502fb9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.990928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.018369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.019298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.020104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.020885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.026041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.026915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.027583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.028204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.029705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.031237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.041889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:19.044199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:19.045256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:19.046333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:19.048042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 315/625] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:19.049029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:19.051167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:19.052268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:19.053269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:19.061715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 311/626] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:19.061715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.109226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.167393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b3bb0-92da-45f2-911d-ddf9261c40c9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.198720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762ab27-f1ce-46cb-98e6-8f64353aba69 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.207693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.208313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.208652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.208928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.209622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:19.210271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.211317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.223738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 316/627] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:19.224672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.247831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.249330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.250101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.250577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.252179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:19.252973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.256031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.263171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.273161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 312/628] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:19.277392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.343494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d03b45-c008-43a9-a273-d22d18fd1b1e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.380997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.382000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.382478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.382961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.383961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:19.384555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.386118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.395406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 317/629] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:19.396592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.399847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.410612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878e85-45e7-439b-9a59-b5da2f91a2bf None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.457224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.457798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.458074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.458344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.466385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.468299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:19.480191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:19.481325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:19.482257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 313/630] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:19.482839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.508920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.513215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ee0d6-87d6-43db-9073-5c4755c5068b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.549786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.550454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.550737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.551015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/dcebfbf0b3084929bc33887467695b00/roles/4981f801f6b14138984b3ae833a5797a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.565414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=dcebfbf0b3084929bc33887467695b00, role_id=4981f801f6b14138984b3ae833a5797a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.566995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.589009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad70fc-fe29-4f30-a411-8b17ea36635a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.621242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687739c5-2d52-4219-bff4-7629028bdfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.621876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687739c5-2d52-4219-bff4-7629028bdfcf None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.622174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687739c5-2d52-4219-bff4-7629028bdfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.622439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687739c5-2d52-4219-bff4-7629028bdfcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.632903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687739c5-2d52-4219-bff4-7629028bdfcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.634347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687739c5-2d52-4219-bff4-7629028bdfcf None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-687739c5-2d52-4219-bff4-7629028bdfcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:19.649202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:19.650166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:19.651057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:19.651057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:19.651057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:19.651057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:19.651057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 314/631] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:19.651650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1fd6-1d2b-4fe0-9b2a-c9d10ee697f4 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.651761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1fd6-1d2b-4fe0-9b2a-c9d10ee697f4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.651977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d1fd6-1d2b-4fe0-9b2a-c9d10ee697f4 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.656418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 315/632] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:19.656418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d1fd6-1d2b-4fe0-9b2a-c9d10ee697f4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.666355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.677258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 318/633] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:19 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/dcebfbf0b3084929bc33887467695b00/roles/4981f801f6b14138984b3ae833a5797a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:19.677258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.775657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d1fd6-1d2b-4fe0-9b2a-c9d10ee697f4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.795788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddde05-ee3a-4490-93f0-a5fcd9422893 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.814400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b64fb-be37-4e22-ad86-cd5059a02163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.815140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b64fb-be37-4e22-ad86-cd5059a02163 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.815423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b64fb-be37-4e22-ad86-cd5059a02163 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.815698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b64fb-be37-4e22-ad86-cd5059a02163 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.816554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b64fb-be37-4e22-ad86-cd5059a02163 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:19.816867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b64fb-be37-4e22-ad86-cd5059a02163 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.818439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b64fb-be37-4e22-ad86-cd5059a02163 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.834322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851c128-67e1-4fad-8236-860fcfcfb214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.835876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c128-67e1-4fad-8236-860fcfcfb214 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.836806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c128-67e1-4fad-8236-860fcfcfb214 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.837090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851c128-67e1-4fad-8236-860fcfcfb214 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.837812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c128-67e1-4fad-8236-860fcfcfb214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:19.839076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c128-67e1-4fad-8236-860fcfcfb214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.839524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851c128-67e1-4fad-8236-860fcfcfb214 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.842476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 316/634] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:19.842476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0caf25-d4f6-4ef7-a202-5dcc29ae39e0 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.842821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0caf25-d4f6-4ef7-a202-5dcc29ae39e0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.845308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0caf25-d4f6-4ef7-a202-5dcc29ae39e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.851754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 319/635] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:19.852912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d18f49-7468-4fcb-9bee-194d6e26ee3b None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.853198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d18f49-7468-4fcb-9bee-194d6e26ee3b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.853471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d18f49-7468-4fcb-9bee-194d6e26ee3b None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.856435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 320/636] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:19.856435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d18f49-7468-4fcb-9bee-194d6e26ee3b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.861714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3c0caf25-d4f6-4ef7-a202-5dcc29ae39e0 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:19.862946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3c0caf25-d4f6-4ef7-a202-5dcc29ae39e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:19.867508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.924142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.940411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 317/637] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:19.940411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0caf25-d4f6-4ef7-a202-5dcc29ae39e0 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.975534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d18f49-7468-4fcb-9bee-194d6e26ee3b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.010662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.011284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.011564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.011835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.046050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0caf25-d4f6-4ef7-a202-5dcc29ae39e0 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.080105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6ec53-885e-4d15-b73d-8683d4812ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.080798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6ec53-885e-4d15-b73d-8683d4812ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.081078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6ec53-885e-4d15-b73d-8683d4812ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.081351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6ec53-885e-4d15-b73d-8683d4812ec2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.086546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:20.088000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:20.090076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6ec53-885e-4d15-b73d-8683d4812ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:20.091499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6ec53-885e-4d15-b73d-8683d4812ec2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:20.098295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-33c6ec53-885e-4d15-b73d-8683d4812ec2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:20.103404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:20.104352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:20.105312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 318/638] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:20.105884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb091263-c201-4fa6-9c22-0b8c1f378c1c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.106550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb091263-c201-4fa6-9c22-0b8c1f378c1c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.106550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb091263-c201-4fa6-9c22-0b8c1f378c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.125172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-cb091263-c201-4fa6-9c22-0b8c1f378c1c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:20.126358 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cb091263-c201-4fa6-9c22-0b8c1f378c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:20.186002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.204660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 321/639] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:20.204660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.210965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 319/640] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:20.213880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb091263-c201-4fa6-9c22-0b8c1f378c1c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.311156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1dd0c0-fba8-4977-9286-773551100047 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.321788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.326343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb091263-c201-4fa6-9c22-0b8c1f378c1c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.346539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.347255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.347605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.348195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.360019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:20.365189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:20.365605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:20.367071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:20.367919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:20.368578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.368578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.368578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.368578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:20.368578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:20.369419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:20.372168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 322/641] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:20.373058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.390116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 320/642] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:20.391131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.437648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.494034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed09b7-a739-4d77-b635-31da1f5b5a4a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.494248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5cf22-9dd8-49cb-8b99-a4a53ecd9942 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.529830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.530462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.530743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.531021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.532287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.533633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.533921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.534198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.541189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:20.545433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:20.546384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:20.547322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:20.548145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:20.548791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:20.548870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 321/643] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:20.548870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:20.565988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:20.567043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:20.568025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:20.568025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:20.568025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:20.568025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:20.568025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 323/644] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:20.568025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.590492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.660315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6200459-1d16-46ea-9fe2-e2976a16154b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.661862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.675899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a901f-00c6-40a0-a9c6-6d921a9472ff None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.696986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.697605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.697882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.698153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.698858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:20.699160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:20.700563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:20.708393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 322/645] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:20.709173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.714551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.715255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.715540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.715818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.791195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:20.792742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:20.817541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.818297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d828a9d-e1c3-4507-b5bc-e4a667424bae None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:20.857546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:20.858167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:20.858449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:20.858724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:20.859440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:20.859752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:20.861218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:20.875907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 323/646] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:20.875907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.906551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 324/647] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:20.906551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:20.910680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:20.988967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853586-15e1-4ea3-9e08-9e090e1a13bb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.011106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310da7f-77ba-4bb3-a563-fd954265d296 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.024605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.025222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.025505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.025778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.026469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:21.026772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.028410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.030787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.044423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 324/648] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:21.045374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.050789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.051661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.051949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.052224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.063302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.064787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:21.079904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.080974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.081841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:21.081841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:21.081841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:21.081841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:21.081841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 325/649] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.081841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.139300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.155055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece1312-9b20-4ff5-bc26-827b893edaa4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.189464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250688c-5ba6-4326-b3ff-4e4d7cf5f84c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.190052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.190687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.190966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.191238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.202007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.203440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:21.218433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.219302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.220155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:21.220155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:21.220155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:21.220155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:21.220155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 325/650] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.220768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.231871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.233751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.234036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.234311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.248131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.249630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:21.264877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.265764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.266638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:21.266638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:21.266638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:21.266638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:21.266638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 326/651] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.266921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.292212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.345305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101409-211c-4afa-97ef-58af71649ae8 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.374945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a728884-fde4-48b7-8096-c4f92738e890 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.379176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.379818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.380095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.380370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.387233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.388704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.390489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:21.400938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.403817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 326/652] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.404705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.420295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.420961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.421254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.421535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.422242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:21.422548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.423924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.438280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 327/653] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:21.439373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.516698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea926f56-aa6d-4322-a1a6-ba0b02a03efd None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.534494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.551947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfa2af-f0d3-4a0b-a206-ebb6e81d38a4 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.559628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.560341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.560745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.561088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.561858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:21.562244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.563745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.576717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 327/654] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:21.581725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.589091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.589765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.590509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.590792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.591498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:21.591805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.594144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.611888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 328/655] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:21.612889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.649898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.695145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde64233-5460-4288-8a14-8ed144c40326 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.718732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7909-ebb0-4563-a246-dfe8d0546fed None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.728625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.729240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.729512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.729779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.730516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:21.730815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.732215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.743889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 328/656] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:21.744821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.756943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.757656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.757933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.758206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.765658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.767088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:21.778348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.779432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 329/657] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.780500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.786934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.844288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098c1fe-6cdf-4a41-a074-3a7af90a7a1a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.861146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.878112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bd3fb-148a-4cbd-87e7-638b9537038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.878883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bd3fb-148a-4cbd-87e7-638b9537038b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.879164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bd3fb-148a-4cbd-87e7-638b9537038b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.879437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bd3fb-148a-4cbd-87e7-638b9537038b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.886156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256c919-8361-4537-8cbc-23b8231a2511 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.887819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bd3fb-148a-4cbd-87e7-638b9537038b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.889345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bd3fb-148a-4cbd-87e7-638b9537038b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-e30bd3fb-148a-4cbd-87e7-638b9537038b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:21.900984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.902054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:21.903153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 329/658] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.903871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f38f77-01d2-4d13-8dd6-3b5df01f7a89 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.903871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f38f77-01d2-4d13-8dd6-3b5df01f7a89 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.904081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f38f77-01d2-4d13-8dd6-3b5df01f7a89 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.905487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 330/659] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:21.906023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f38f77-01d2-4d13-8dd6-3b5df01f7a89 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.923955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.924717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.924927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.925201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.932545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.933993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:21.945749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.946780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 330/660] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.947680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.008997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f38f77-01d2-4d13-8dd6-3b5df01f7a89 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.013920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.041706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.042313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.042617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.043200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.043988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=784880e276bf4d64ae785609a463403a) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.044296 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.045751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.050773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc939-e08c-4e2f-8ca8-33e8b2438277 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.051334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 331/661] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/role_assignments?user.id=784880e276bf4d64ae785609a463403a&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.052100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.096831 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd85a9-cab6-441d-b508-88e5d6c83f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.097376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd85a9-cab6-441d-b508-88e5d6c83f70 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.097716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd85a9-cab6-441d-b508-88e5d6c83f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.097992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd85a9-cab6-441d-b508-88e5d6c83f70 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.099164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd85a9-cab6-441d-b508-88e5d6c83f70 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.099164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd85a9-cab6-441d-b508-88e5d6c83f70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.101167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd85a9-cab6-441d-b508-88e5d6c83f70 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.105490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.129346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 331/662] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.129346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dc2ea-ce13-45c8-b3ed-6b4c5f65a1cd None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.129346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dc2ea-ce13-45c8-b3ed-6b4c5f65a1cd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.129346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dc2ea-ce13-45c8-b3ed-6b4c5f65a1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.147079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-af4dc2ea-ce13-45c8-b3ed-6b4c5f65a1cd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:22.148299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-af4dc2ea-ce13-45c8-b3ed-6b4c5f65a1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:22.182014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccffb5e-42ab-486c-b46b-f81f9f403f80 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.221979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.222601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.222884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.223159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.223954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.224263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.226258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.230349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 332/663] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:22.231191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dc2ea-ce13-45c8-b3ed-6b4c5f65a1cd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.249418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 332/664] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.250194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.299611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.351435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dc2ea-ce13-45c8-b3ed-6b4c5f65a1cd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.368218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.371403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef269-e240-4c0d-90dd-ecc0f75238d7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.387042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.387663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.387943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.388214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.389005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.389524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.390729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.403960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 333/665] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.404823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.414518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.415218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.415500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.415773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.495034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.496517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.512585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f221bb-ad60-474f-856d-5d5d4c0c1fd2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.526513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.548782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.549432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.549670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.552143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.601192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.615285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 333/666] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:22.615285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.626404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.627858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.707205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509d03-a5b2-460d-9fc8-3e941d93253f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.734762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 334/667] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:22.735823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.746922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.747570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.747852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.748126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.750205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.761853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:22.767579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.768723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:22.769781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:22.770701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 334/668] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.776197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.835904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.848849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f79c-abc2-4678-bdb9-77c5c5b2d953 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.883654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.884191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.884501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.884777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.884992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7869f2-b84d-4d2b-a715-4d1550acefe4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.885577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=813427e9e85a4b48a550a290211f39b5) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.885887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.887310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.893269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 335/669] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/role_assignments?user.id=813427e9e85a4b48a550a290211f39b5&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.894059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.935805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.936611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.936962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.937327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.949887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.951445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:22.967639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.968770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.969845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:22.969845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:22.969845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.969845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:22.975309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 335/670] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.976419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.001676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.034927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ed806-7ebd-4936-9dc3-1f905d28b53e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.086195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.088477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.088600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fbb69-2cce-424b-903c-0747d761539b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.089038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.089467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.105225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.130342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.131543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.131839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.132082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.132881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.133178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.134730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.145056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 336/671] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.146031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.235075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.236999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:23.257444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.259658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:23.259658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:23.259658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.259658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:23.265518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 336/672] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.265518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.279244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e413135-cad1-48f8-951d-aba5f1e7a7d1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.324429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.324429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.324429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.324429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.325433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.325835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.327322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.351631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 337/673] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.352815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.386485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69dbf-8590-4fc0-a815-7ac1c362fb16 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.397977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.418144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.428253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.429488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.429773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.430052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.445157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.446616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:23.468752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.469676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.470531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:23.470531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:23.470531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.470531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:23.472862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 337/674] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.474356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.476468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc3f2b-6c6a-4a4a-82bb-90e0a8389c04 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.515245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.516114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.516532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.516912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.517706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.518112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.519693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.536974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 338/675] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.538087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.596466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669644d-ea5c-41c6-8507-8f79df1975f4 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.633048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.633832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.634219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.634597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.642689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.644224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:23.656337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.657410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:23.658478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:23.665172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 338/676] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.666121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.666411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.675109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b800a-adb9-4aad-a8e7-a694a77e18b7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.712937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.722574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.723432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.723860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.724924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.726239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.726696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.728711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.752239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 339/677] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.753003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.786041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887def1b-6480-43c0-9bb9-94be41d49a2f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.830094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.831519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.832010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.832543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.841911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.843588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.856935 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff490465-0790-4975-91f0-7a3feef462e1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:23.859995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.860863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:23.861727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:23.864066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 339/678] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.865803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.897550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.898089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.898370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.898645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.899508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.899821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.901281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.919661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 340/679] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.920597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.937938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.980602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.983938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986a7ee-88e7-4fe4-a02a-782426daa6ca None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.017376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.018162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.018601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.018919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.028188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.029768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.030132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf292ff-c7b0-43d0-babc-449740f98062 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:24.042770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:24.043985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:24.045182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:24.046878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 340/680] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:24.047840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.071976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.072914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.073488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.073912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.074811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.075264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.076897 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.095425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 341/681] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.096541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.156888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b37618-34b3-41cc-910b-7912c10d2b88 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.184482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.194835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.195378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.195654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.195925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.196785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=78776208cde94134b478ceee6cc70ec8) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.197243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.208555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.208555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 341/682] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/role_assignments?user.id=78776208cde94134b478ceee6cc70ec8&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.208555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.221664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a064138-5f6e-4480-ac52-1042304da00b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.236557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.275534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf3aab-eb80-40c4-bc69-57d46394fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.276658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf3aab-eb80-40c4-bc69-57d46394fa42 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.276949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf3aab-eb80-40c4-bc69-57d46394fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.277228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf3aab-eb80-40c4-bc69-57d46394fa42 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.278335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf3aab-eb80-40c4-bc69-57d46394fa42 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.278652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf3aab-eb80-40c4-bc69-57d46394fa42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.281176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf3aab-eb80-40c4-bc69-57d46394fa42 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.307762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 342/683] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.309501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5214-a3d5-4d5f-b41a-70e72d834cc1 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.309788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5214-a3d5-4d5f-b41a-70e72d834cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.310066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e5214-a3d5-4d5f-b41a-70e72d834cc1 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.312103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 343/684] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:24.312960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e5214-a3d5-4d5f-b41a-70e72d834cc1 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.334653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d499ba1-534c-47e7-9a14-b107583da4cb None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.376553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71833f52-2bae-4dfa-9380-8d5624225128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.377551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71833f52-2bae-4dfa-9380-8d5624225128 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.378019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71833f52-2bae-4dfa-9380-8d5624225128 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.378446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71833f52-2bae-4dfa-9380-8d5624225128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.379522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71833f52-2bae-4dfa-9380-8d5624225128 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=54aedfbb64c54c589003b335fde2a17b) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.379995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71833f52-2bae-4dfa-9380-8d5624225128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.382140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71833f52-2bae-4dfa-9380-8d5624225128 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.390368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 342/685] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/role_assignments?user.id=54aedfbb64c54c589003b335fde2a17b&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.391875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349141de-458b-4168-a677-a2f106b45e26 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.392159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349141de-458b-4168-a677-a2f106b45e26 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.392609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349141de-458b-4168-a677-a2f106b45e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.416997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-349141de-458b-4168-a677-a2f106b45e26 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:24.418553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-349141de-458b-4168-a677-a2f106b45e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:24.438293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e5214-a3d5-4d5f-b41a-70e72d834cc1 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.488285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.495731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.506237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.506237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.506237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.527797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 343/686] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:24.551309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349141de-458b-4168-a677-a2f106b45e26 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.584164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.585028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.586680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.675007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349141de-458b-4168-a677-a2f106b45e26 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.711025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 344/687] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:24.711592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.712776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.712776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.712776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.719806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.733141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:24.741332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:24.742240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:24.743056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:24.750978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 344/688] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:24.752820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.762097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.840619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692bf88-166f-4fbb-a3e6-0da1c9190ec1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.855096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.883405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.883645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d352f4-97cd-40f5-ab3d-1f43f7e44bc5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.884524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.884969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.885386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.901283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.902952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:24.918800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:24.919918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:24.921007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:24.921007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:24.921007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:24.921007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:24.922841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 345/689] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:24.932483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.932483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.932483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.932483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.932913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.932913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.932913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.934544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.947474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 345/690] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.953625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.059901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.077579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c57d4d-7b09-49b1-a238-bb8deee0a4f2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.128801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.129427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.129702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.129974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.130706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.131007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.132390 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.136482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3f7b-ffc1-480d-8482-f13b4ce980cd None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.146519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 346/691] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.151965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.204513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0435ce-b8cf-4e9d-b906-d9e104ad9a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.205529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0435ce-b8cf-4e9d-b906-d9e104ad9a7c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.206011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0435ce-b8cf-4e9d-b906-d9e104ad9a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.206485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0435ce-b8cf-4e9d-b906-d9e104ad9a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.226232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.229161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0435ce-b8cf-4e9d-b906-d9e104ad9a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.236199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0435ce-b8cf-4e9d-b906-d9e104ad9a7c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-6d0435ce-b8cf-4e9d-b906-d9e104ad9a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:25.262422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:25.263336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:25.264273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:25.264273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:25.264273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:25.264273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:25.267623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 346/692] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:25.273485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdf4f8-30eb-47bf-9a87-3fd67cfdb2a0 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.274359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdf4f8-30eb-47bf-9a87-3fd67cfdb2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.274842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdf4f8-30eb-47bf-9a87-3fd67cfdb2a0 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.276582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 347/693] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:25.277478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdf4f8-30eb-47bf-9a87-3fd67cfdb2a0 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.280837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c0a01-1ee5-4fbd-aaba-88df9c3a943d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.319464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c4038-bce5-432f-b23b-d50ddd805cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.320450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4038-bce5-432f-b23b-d50ddd805cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.321041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4038-bce5-432f-b23b-d50ddd805cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.321515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4038-bce5-432f-b23b-d50ddd805cc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.328985 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c4038-bce5-432f-b23b-d50ddd805cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.330651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c4038-bce5-432f-b23b-d50ddd805cc5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-274c4038-bce5-432f-b23b-d50ddd805cc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:25.349567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:25.350459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:25.351322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 347/694] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:25.356962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a371a-0003-4a8b-9dd0-aafd6b6a0844 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.357244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a371a-0003-4a8b-9dd0-aafd6b6a0844 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.357512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a371a-0003-4a8b-9dd0-aafd6b6a0844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.365926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.378131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bd5a371a-0003-4a8b-9dd0-aafd6b6a0844 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:25.387176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bd5a371a-0003-4a8b-9dd0-aafd6b6a0844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:25.408897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdf4f8-30eb-47bf-9a87-3fd67cfdb2a0 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.451465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.452069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.452350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.452655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.453419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.453646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.454049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 348/695] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:25.455071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.455333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a371a-0003-4a8b-9dd0-aafd6b6a0844 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.492117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 348/696] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.493441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.556059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.594365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a371a-0003-4a8b-9dd0-aafd6b6a0844 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.612776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa17615-50b8-4e90-ad8f-0473caa55c2d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.630944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.631592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.632022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.632227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.633057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.633362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.634952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.636849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.650023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 349/697] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.650851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.654369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.655248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.655801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.656327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.669632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.671596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:25.688392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:25.689706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:25.690616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:25.690616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:25.690616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:25.690616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:25.697410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 349/698] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:25.698551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.762903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d81e-7b66-405e-a85c-5f8f3f0f5580 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.799313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.800131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.800318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.800628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.808688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.809919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0801b2-d854-4482-9fbb-02753f573910 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.810107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.813298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:25.823418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:25.824601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:25.825722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:25.832049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 350/699] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:25.833036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.852003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.852874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.853257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.853643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.854690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=dcebfbf0b3084929bc33887467695b00) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.855114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.856820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.872580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 350/700] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/role_assignments?user.id=dcebfbf0b3084929bc33887467695b00&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.873922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.898019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.959259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845832b-eb78-4e1c-bc3b-c69c7d4ee772 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.011726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72a69d-39a1-4fb7-b2ae-76624e54d037 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.029155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.030141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.030683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.031196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.032538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.033064 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.034771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.052401 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.053278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.053683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.058437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.058437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.058437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.058437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.064527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 351/701] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:26.072949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.088049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 351/702] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:26.091322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.146653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.191857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88e6b2-7c9f-44f9-979c-0e9108e65986 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.208094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.229228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.229844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.230125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.230397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.231100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.236407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.274445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118f537-ea9f-488d-beaa-482a3fdf3720 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.309305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1fc5c-f915-4cd5-a2dd-80074874095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.309849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1fc5c-f915-4cd5-a2dd-80074874095f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.310129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1fc5c-f915-4cd5-a2dd-80074874095f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.310391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1fc5c-f915-4cd5-a2dd-80074874095f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.398000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1fc5c-f915-4cd5-a2dd-80074874095f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.399468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1fc5c-f915-4cd5-a2dd-80074874095f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.429087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.437758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 352/703] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:26 2022] POST /identity/v3/users => generated 254 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:26.534734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 352/704] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:26.540461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.632692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e2b75-7a69-488c-b67a-4095f05f451d None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.632997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e2b75-7a69-488c-b67a-4095f05f451d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.633216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e2b75-7a69-488c-b67a-4095f05f451d None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.634646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 353/705] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:26.654202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605dd2-cb5d-4afc-90df-a12fd8fa2c22 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.654525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605dd2-cb5d-4afc-90df-a12fd8fa2c22 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.654750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605dd2-cb5d-4afc-90df-a12fd8fa2c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.669243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3b075-5654-4a65-9e76-92b6f2366af3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.679484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fe605dd2-cb5d-4afc-90df-a12fd8fa2c22 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:26.681095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fe605dd2-cb5d-4afc-90df-a12fd8fa2c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:26.711693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.712340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.712714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.712935 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.724926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.726376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:26.742181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:26.743242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:26.744858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:26.744858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:26.744858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:26.744858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:26.744858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 353/706] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:26.749136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.770890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 354/707] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:26.776318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.794257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe605dd2-cb5d-4afc-90df-a12fd8fa2c22 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.877325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7980bb-900e-42fd-bb44-7047f6022734 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.880780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.917742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac927c-8721-46a9-8b20-3463cfefc809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.918666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac927c-8721-46a9-8b20-3463cfefc809 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.919674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac927c-8721-46a9-8b20-3463cfefc809 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.920153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac927c-8721-46a9-8b20-3463cfefc809 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.921385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac927c-8721-46a9-8b20-3463cfefc809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.922121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac927c-8721-46a9-8b20-3463cfefc809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.924395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac927c-8721-46a9-8b20-3463cfefc809 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.933279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe605dd2-cb5d-4afc-90df-a12fd8fa2c22 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.941754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 354/708] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:26.943375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fb807-be59-4fe0-83cd-c831ce536d09 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.943878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fb807-be59-4fe0-83cd-c831ce536d09 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.944317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fb807-be59-4fe0-83cd-c831ce536d09 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.945974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 355/709] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:26.947046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fb807-be59-4fe0-83cd-c831ce536d09 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.991593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef155926-a8ba-47cf-8cd8-11dfc24dca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.992621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155926-a8ba-47cf-8cd8-11dfc24dca72 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.993098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155926-a8ba-47cf-8cd8-11dfc24dca72 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.993581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155926-a8ba-47cf-8cd8-11dfc24dca72 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.994411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef155926-a8ba-47cf-8cd8-11dfc24dca72 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.995963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef155926-a8ba-47cf-8cd8-11dfc24dca72 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ef155926-a8ba-47cf-8cd8-11dfc24dca72 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:27.007801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:27.009022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:27.009868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:27.012733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 355/710] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:27.014312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62a96-8c0d-4ab9-ae4a-fd18806c21f7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.014815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62a96-8c0d-4ab9-ae4a-fd18806c21f7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.015294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62a96-8c0d-4ab9-ae4a-fd18806c21f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.035615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-17f62a96-8c0d-4ab9-ae4a-fd18806c21f7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:27.037729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-17f62a96-8c0d-4ab9-ae4a-fd18806c21f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:27.064958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.091393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80fb807-be59-4fe0-83cd-c831ce536d09 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.110837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 356/711] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:27.112149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62a96-8c0d-4ab9-ae4a-fd18806c21f7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.144042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.146281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.146578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.146858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.155742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.157807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:27.175288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:27.176295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:27.177468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:27.180525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 356/712] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:27.181947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.194374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.251448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62a96-8c0d-4ab9-ae4a-fd18806c21f7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.293851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.295094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.295962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.296522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.297624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.298142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.300522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.308109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 357/713] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.309967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.312960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b7070-f506-4bed-80b3-5136355c5c0f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.356699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.357213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.357499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.357762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] PATH_INFO: `/v3/services/network` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.358390 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.359757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:27.371426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:27.372477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:27.373306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:27.377743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 357/714] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:27.378601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.392172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.454195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7fcb1-dc1b-44b3-8c2f-0b8100155cb0 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.472756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.496894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.497811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.498484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.499016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.500450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.501093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.502809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffd121-43e8-424d-bc2a-2bfac8a73e65 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.503217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.520418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 358/715] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.520418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.543424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.544455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.544987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.545472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.546382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.546938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.548587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.555103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 358/716] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.556224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.632774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.636711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abba16-0bc9-4e3c-8c58-161e60d49564 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.668464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3854b8-a306-4130-a449-05acb3a22092 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.677613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.678237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.678518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.678790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.679477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] RBAC: Adding query filter params (name=network) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.679780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.681295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.688435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 359/717] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.693558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.717541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df9f70-e38f-40be-b02b-84d63e1fa204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.718491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df9f70-e38f-40be-b02b-84d63e1fa204 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.718777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df9f70-e38f-40be-b02b-84d63e1fa204 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.719056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df9f70-e38f-40be-b02b-84d63e1fa204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.719865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df9f70-e38f-40be-b02b-84d63e1fa204 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=784880e276bf4d64ae785609a463403a) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.720177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df9f70-e38f-40be-b02b-84d63e1fa204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.722212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df9f70-e38f-40be-b02b-84d63e1fa204 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.730453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 359/718] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/role_assignments?user.id=784880e276bf4d64ae785609a463403a&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.763960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.848619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d290b1-ce80-4a00-bcc5-708d9a8e860a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.888957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05198acb-da7b-49a1-9401-33d9c3d39470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.889603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05198acb-da7b-49a1-9401-33d9c3d39470 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.889886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05198acb-da7b-49a1-9401-33d9c3d39470 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.890161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05198acb-da7b-49a1-9401-33d9c3d39470 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.890872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05198acb-da7b-49a1-9401-33d9c3d39470 None admin] RBAC: Adding query filter params (type=network) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.891181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05198acb-da7b-49a1-9401-33d9c3d39470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.892648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05198acb-da7b-49a1-9401-33d9c3d39470 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.901949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 360/719] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:28.389975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdfdc-a883-481e-bb94-4961546d6404 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.389975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdfdc-a883-481e-bb94-4961546d6404 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.389975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cdfdc-a883-481e-bb94-4961546d6404 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.390998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 360/720] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:28.396692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d368-0f7f-4c7e-b0bd-a9584bd5af27 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.397221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d368-0f7f-4c7e-b0bd-a9584bd5af27 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.397722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d368-0f7f-4c7e-b0bd-a9584bd5af27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.416857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-0df1d368-0f7f-4c7e-b0bd-a9584bd5af27 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:28.418399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0df1d368-0f7f-4c7e-b0bd-a9584bd5af27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:28.492181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 361/721] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:28.539880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdfdc-a883-481e-bb94-4961546d6404 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.668706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cdfdc-a883-481e-bb94-4961546d6404 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:28.717104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54151a-8dea-496a-9f0e-7da24507c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:28.717801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54151a-8dea-496a-9f0e-7da24507c073 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.718090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54151a-8dea-496a-9f0e-7da24507c073 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.718371 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54151a-8dea-496a-9f0e-7da24507c073 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.720440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:28.736495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54151a-8dea-496a-9f0e-7da24507c073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-9a54151a-8dea-496a-9f0e-7da24507c073 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:28.742221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:28.743128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:28.743958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 361/722] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:28.748746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1d368-0f7f-4c7e-b0bd-a9584bd5af27 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.800257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:28.886994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1d368-0f7f-4c7e-b0bd-a9584bd5af27 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:28.925777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93757e2c-9183-4a1a-bedb-1c6d265757ed None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.926020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93757e2c-9183-4a1a-bedb-1c6d265757ed None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.926255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93757e2c-9183-4a1a-bedb-1c6d265757ed None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.927781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 362/723] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:28.935751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105d5b0-d360-45f7-8390-d864ae518f24 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.936031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105d5b0-d360-45f7-8390-d864ae518f24 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.936311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105d5b0-d360-45f7-8390-d864ae518f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.937226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:28.938071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.938555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.939149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.940080 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:28.940629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:28.942734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:28.951597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 362/724] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:28.956797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.961703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-2105d5b0-d360-45f7-8390-d864ae518f24 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:28.962898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2105d5b0-d360-45f7-8390-d864ae518f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:29.100755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.104479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 363/725] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:29.109729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e89a34-a029-4167-b36c-fdce9780768e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.144006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.144783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.146383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105d5b0-d360-45f7-8390-d864ae518f24 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.149463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.149745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.174581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.182792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:29.194497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.196342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:29.196342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:29.196342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:29.196342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:29.196342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 363/726] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:29.201324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.308644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b83f89-8921-47e7-be9c-e091ca8bcc72 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.349305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.350728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.351017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.351294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.351996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:29.352302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.353699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.367374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 364/727] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:29.372648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.383429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.454482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105d5b0-d360-45f7-8390-d864ae518f24 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.487571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.496005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.501520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.501520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.501520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.510339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:29.516022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:29.518229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 364/728] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:29.525193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.534388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88b2c3-bca3-489e-9fb1-d468e471d38a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.633386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd9251-c848-49c5-8dee-18b5e720a04e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.669974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.670318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.670627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.670813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.670978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.671130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.671286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.671437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.672090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:29.672412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.673966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.683660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 365/729] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:29.685633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.687168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.689633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:29.713036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.714054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:29.715032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:29.720547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 365/730] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:29.725788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.748305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.804810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed6f88-1d3e-4982-a978-7482bb1e91c4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.841236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.841895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.842181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.842729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.855261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.857272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:29.874138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.875102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.875967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:29.875967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:29.875967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:29.875967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:29.875967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 366/731] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:29.889454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.908696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.982541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf3c18-f5f1-4cfc-94b1-56c8ee88e72b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.999759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.004896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd2943-5584-42f4-aaa9-8e35c2923cef None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.051169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.052234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.052773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.053968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.055649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.056321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.059048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.067225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.070412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.070979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.073149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.074079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.074596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.076205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.090255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 367/732] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.095704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 366/733] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:30.096865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.104511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.211816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f742e-ec41-490f-b306-991cfea610a5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.224035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2570e4-f8bb-42e1-8378-5f5431a3c32d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.236179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.248317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fc128-e973-464b-b931-b9adc1a3be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.254729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc128-e973-464b-b931-b9adc1a3be15 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.254729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc128-e973-464b-b931-b9adc1a3be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.254729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc128-e973-464b-b931-b9adc1a3be15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.263682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67207f3-7e95-450b-ac1f-4f978f0bfe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.264328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67207f3-7e95-450b-ac1f-4f978f0bfe4a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.264642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67207f3-7e95-450b-ac1f-4f978f0bfe4a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.264922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67207f3-7e95-450b-ac1f-4f978f0bfe4a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.273690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67207f3-7e95-450b-ac1f-4f978f0bfe4a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.275260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67207f3-7e95-450b-ac1f-4f978f0bfe4a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.325125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fc128-e973-464b-b931-b9adc1a3be15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.326621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fc128-e973-464b-b931-b9adc1a3be15 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-a67207f3-7e95-450b-ac1f-4f978f0bfe4a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:30.405829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:30.406953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 368/734] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed3ef5-e99d-47bc-b809-87c43feea1e7 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.413758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed3ef5-e99d-47bc-b809-87c43feea1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.414473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed3ef5-e99d-47bc-b809-87c43feea1e7 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.414473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 369/735] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:30.414473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f43d1-70e7-4c3e-af95-6aacef7127ea None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.414473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f43d1-70e7-4c3e-af95-6aacef7127ea None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.414473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f43d1-70e7-4c3e-af95-6aacef7127ea None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.415749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 370/736] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:30.420913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f43d1-70e7-4c3e-af95-6aacef7127ea None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.440709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.450166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 367/737] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:55:30.450166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a392a4-e303-4b3e-a579-c28735b4507a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.450166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a392a4-e303-4b3e-a579-c28735b4507a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.450166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a392a4-e303-4b3e-a579-c28735b4507a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.465707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-63a392a4-e303-4b3e-a579-c28735b4507a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:30.466900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-63a392a4-e303-4b3e-a579-c28735b4507a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:30.484987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.528219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f43d1-70e7-4c3e-af95-6aacef7127ea None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.532955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 368/738] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:30.534163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7869ea-af17-46ec-a302-1a1843b90b38 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.534447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7869ea-af17-46ec-a302-1a1843b90b38 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.534716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7869ea-af17-46ec-a302-1a1843b90b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.561836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7a7869ea-af17-46ec-a302-1a1843b90b38 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:30.563020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7a7869ea-af17-46ec-a302-1a1843b90b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:30.564564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.565185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.565463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.565741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.566560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.566869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.568272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.585147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 371/739] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.585773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.629318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 369/740] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:30.630701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62432a83-8752-42b2-9d7b-555be22e6ef0 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.630997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62432a83-8752-42b2-9d7b-555be22e6ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.631274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62432a83-8752-42b2-9d7b-555be22e6ef0 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.636551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 370/741] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:30.636551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62432a83-8752-42b2-9d7b-555be22e6ef0 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.668404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.690268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118035e2-daf1-4de5-b557-253d7155c061 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.724688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.725321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.725597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.725869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.726441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.735821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.737318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.742722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62432a83-8752-42b2-9d7b-555be22e6ef0 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:30.752051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:30.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:30.753889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:30.753889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:30.753889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.753889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:30.753889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 372/742] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:30.754199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.780133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bd47a-e68b-487f-82c6-73b276ed5ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.780706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd47a-e68b-487f-82c6-73b276ed5ad6 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.780982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd47a-e68b-487f-82c6-73b276ed5ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.781256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd47a-e68b-487f-82c6-73b276ed5ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.781947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bd47a-e68b-487f-82c6-73b276ed5ad6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.783295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bd47a-e68b-487f-82c6-73b276ed5ad6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.865679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249e06f-7ef4-4aeb-afae-ac824b3f2709 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.883523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 371/743] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:30 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:30.887287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096516d3-636f-4053-b89b-14a4b7f8287e None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.887287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096516d3-636f-4053-b89b-14a4b7f8287e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.887287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096516d3-636f-4053-b89b-14a4b7f8287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.894984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.904899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-096516d3-636f-4053-b89b-14a4b7f8287e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:30.906137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-096516d3-636f-4053-b89b-14a4b7f8287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:30.918686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.919330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.919613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.919888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.000419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 372/744] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:31.000419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096516d3-636f-4053-b89b-14a4b7f8287e None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.016164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.019321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.106465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.131632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096516d3-636f-4053-b89b-14a4b7f8287e None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.143635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 373/745] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:31.143635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.167753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.175675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.177034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.177322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.177593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.191248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:31.197351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.198276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 373/746] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.199312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.253816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870c45f-827e-4986-8c09-e7b6489c0831 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.290349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.291311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.291592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.291878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.292574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.292876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.294341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.309842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25bbab-e819-44ef-844e-2960df7f4834 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.311321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 374/747] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:31.312180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.346194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.346817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.347089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.347361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.348631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.349996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.353514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.403251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.419256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a7a0e-b576-4c49-ab6a-2a5136ebcf40 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.437916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 374/748] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:31 2022] POST /identity/v3/services => generated 240 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:31.439067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.464560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.465282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.465604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.465876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.479708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.481174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:31.497788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.498763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.499632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:31.499632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:31.499632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.499632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:31.499632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 375/749] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.499956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.596601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913b060-0b8e-44be-868e-0b3c7b80639d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.639248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.639881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.640167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.640479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.641848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.641848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.642873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.643931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc10a1-00de-4748-a8da-00be073ed3a3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.655812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 375/750] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:31.657002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.690062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.696128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.696655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.696933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.697201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.704119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.705586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:31.717973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.718874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:31.719780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 376/751] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.720253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.730710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.778989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59556b7-ff3a-4379-96a1-295d60bcba47 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.817125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.818147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.818705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.819220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.820281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.820820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.822482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.828806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43039d7-d8a2-4429-9c08-26c79db018d3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.837111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 376/752] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:31.838355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.863290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.863928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.864236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.864636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.875456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.877063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:31.891542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.892470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.893360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:31.893360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:31.893360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.893360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:31.897903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 377/753] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.898761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.951167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963089ab-4f21-410d-91e4-b8bc28cb6e1c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.953876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.989812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.990803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.991327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.991797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.992878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.993400 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.995184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.002443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.008870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df565b7-7e3f-41fd-846d-a10cf2118b1f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.009151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 377/754] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.010162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.052851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.053505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.053778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.054045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.061264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.062713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:32.075010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:32.076055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:32.076977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:32.077433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 378/755] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:32.078380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.130851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18826b2-43fd-4c72-812f-730cb03d4a11 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.178878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.179914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.180441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.180923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.181436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045753a3-e1f0-4b5e-bf2b-5a2ad8fd2ab3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.182013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.182496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.184913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.185029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.203260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 378/756] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.204520 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.223742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.224365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.224999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.225890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.226522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=dcebfbf0b3084929bc33887467695b00) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.226942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.228613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.241175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 379/757] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/role_assignments?user.id=dcebfbf0b3084929bc33887467695b00&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.242281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.262304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.346662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2850-98b9-4f3a-9aa4-44829b63126a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.389557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.391066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.391585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.392079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.393098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.393737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.395961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.410802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 379/758] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.416863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.494674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a46c9b-5974-4801-bce1-1dd11dbdd88d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.549392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.554485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602c789-51da-4317-b311-b0b27bab429e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.565129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.565797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.566620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.566853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.589340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.589958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.590647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.590754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.590965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.591244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.591980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=f247bde98344474ea5a62b4fa04464b8) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.592286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.594717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.600913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 380/759] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/role_assignments?user.id=f247bde98344474ea5a62b4fa04464b8&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:32.611058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:32.612049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:32.613033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:32.616080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 380/760] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:32.618549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.744978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef83c40-918e-4533-b791-8f35f99e8b89 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.792460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079acd2-6306-4f59-bbb5-75f504e4fb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.793100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079acd2-6306-4f59-bbb5-75f504e4fb2f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.793430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079acd2-6306-4f59-bbb5-75f504e4fb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.793707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079acd2-6306-4f59-bbb5-75f504e4fb2f None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.794522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079acd2-6306-4f59-bbb5-75f504e4fb2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.794829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079acd2-6306-4f59-bbb5-75f504e4fb2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.796267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3079acd2-6306-4f59-bbb5-75f504e4fb2f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.821214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 381/761] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.826558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.867747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.958186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513628b-f45d-4498-a7f6-8698bcc174eb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.992888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410217aa-208e-4d77-bfc9-4301598096de None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.993226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410217aa-208e-4d77-bfc9-4301598096de None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.993450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410217aa-208e-4d77-bfc9-4301598096de None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.994938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.995638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.995865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.996132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 382/762] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:32.996235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.000145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b535-f657-419e-8573-c9029af1f59b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.000465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b535-f657-419e-8573-c9029af1f59b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.000751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b535-f657-419e-8573-c9029af1f59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.009213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.010669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.019809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a872b535-f657-419e-8573-c9029af1f59b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:33.021163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a872b535-f657-419e-8573-c9029af1f59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:33.025723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:33.026579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:33.027467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:33.027467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:33.027467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:33.027467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:33.027467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 381/763] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:33.032354 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.055053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.169446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 383/764] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:33.176121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.196063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16e776-11f6-48e2-8276-cf71ae3d7ba5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.208211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b535-f657-419e-8573-c9029af1f59b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.280035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.280592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.280866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.281132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.281821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.282122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.283475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.314949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 382/765] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.322053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.362084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b535-f657-419e-8573-c9029af1f59b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.401432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.402072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.402350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.402618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.403249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.404628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:33.424354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:33.425887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:33.426718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:33.432530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 384/766] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:33.439974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.455494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.523467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884f3c7-5936-42e3-931e-9127128352eb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.546362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca256d98-6eb3-4dd6-af13-bbaac8630247 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.582765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.583762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.584332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.584839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.585716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.586215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.587774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.593162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 385/767] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.600086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.606075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.606742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.607022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.607300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.619889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.621442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:33.644107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:33.645072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:33.645920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:33.648697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.652852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 383/768] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:33.660634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.694066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.717250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa0956-a03e-4352-ba2d-fb4989386b28 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.757543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c991d7-51de-48bb-a965-803cfeaa5b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.758477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c991d7-51de-48bb-a965-803cfeaa5b2b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.758766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c991d7-51de-48bb-a965-803cfeaa5b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.759093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c991d7-51de-48bb-a965-803cfeaa5b2b None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.760047 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c991d7-51de-48bb-a965-803cfeaa5b2b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.760357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c991d7-51de-48bb-a965-803cfeaa5b2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.761927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c991d7-51de-48bb-a965-803cfeaa5b2b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.768355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 386/769] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.770818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0add-17cf-4f1d-842c-648dfd28da0d None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.771115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0add-17cf-4f1d-842c-648dfd28da0d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.771386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0add-17cf-4f1d-842c-648dfd28da0d None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.776800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 387/770] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:33.776800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce0add-17cf-4f1d-842c-648dfd28da0d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.784140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd5508-7f07-4e9c-babf-a896a5a521e9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.827375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfbe1f-0df6-492c-b663-b4b6550ee2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.827989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfbe1f-0df6-492c-b663-b4b6550ee2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.828227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfbe1f-0df6-492c-b663-b4b6550ee2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.828468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfbe1f-0df6-492c-b663-b4b6550ee2f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.829217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfbe1f-0df6-492c-b663-b4b6550ee2f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.829523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfbe1f-0df6-492c-b663-b4b6550ee2f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.830939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfbe1f-0df6-492c-b663-b4b6550ee2f3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.852485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 384/771] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.853852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa4e34-6a51-497e-81ab-1d52253c6a35 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.854190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa4e34-6a51-497e-81ab-1d52253c6a35 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.854474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa4e34-6a51-497e-81ab-1d52253c6a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.872593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-eaaa4e34-6a51-497e-81ab-1d52253c6a35 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:33.873799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-eaaa4e34-6a51-497e-81ab-1d52253c6a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:33.904862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce0add-17cf-4f1d-842c-648dfd28da0d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.932467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.940203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.941009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.947739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.947739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.947739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] RBAC: Adding query filter params (interface=public, service_id=3296f883d9ea4557ab312de331e2522c, region_id=RegionOne) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.947739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.947739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.948192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 385/772] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:33.948192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa4e34-6a51-497e-81ab-1d52253c6a35 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.951886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 388/773] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/endpoints?service_id=3296f883d9ea4557ab312de331e2522c&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.957539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.975980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.072108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa4e34-6a51-497e-81ab-1d52253c6a35 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.083667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8b9e5-83b3-42ee-bf39-34da1d0d6389 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.109559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.110396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.110804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.111188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/784880e276bf4d64ae785609a463403a/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.113303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=784880e276bf4d64ae785609a463403a, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.114828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.125297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.125910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.126211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.126484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.131440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.131440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.134530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 389/774] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:34.135299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.197728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 386/775] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:33 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/784880e276bf4d64ae785609a463403a/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:34.233229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.273320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecb517-f9e6-437c-b037-2e5a6f9c7e7d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.318373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.319038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.319315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.319585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.320226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.321732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:34.340491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:34.341496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:34.342311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:34.345186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 390/776] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:34.350137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.444768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.490931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d26ab-98f1-4d03-9aec-da03df36f5bb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.527948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160e97b-667b-4231-9258-16d2443b7061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.528603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160e97b-667b-4231-9258-16d2443b7061 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.528881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160e97b-667b-4231-9258-16d2443b7061 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.529148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160e97b-667b-4231-9258-16d2443b7061 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.529825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160e97b-667b-4231-9258-16d2443b7061 None admin] RBAC: Adding query filter params (name=network) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:34.530122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160e97b-667b-4231-9258-16d2443b7061 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.531465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160e97b-667b-4231-9258-16d2443b7061 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.536531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 387/777] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:34.542035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.728260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13c263-a5cd-42c8-9099-564fd3ee61f3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.762509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.768603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.769251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.769530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.769801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.770490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] RBAC: Adding query filter params (type=network) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:34.773257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.773257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.780055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b302a-544b-4346-922b-38549a06ec07 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.780480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b302a-544b-4346-922b-38549a06ec07 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.780642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 391/778] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:34.780906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b302a-544b-4346-922b-38549a06ec07 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.782547 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 388/779] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:34.785928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.787907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d09a45-d100-45dc-b601-8a8004155a23 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.788254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d09a45-d100-45dc-b601-8a8004155a23 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.788648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d09a45-d100-45dc-b601-8a8004155a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.814167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-11d09a45-d100-45dc-b601-8a8004155a23 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:34.815350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-11d09a45-d100-45dc-b601-8a8004155a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:34.920457 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 389/780] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:34.951573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d09a45-d100-45dc-b601-8a8004155a23 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.977603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b7848-8795-41bb-8cd2-b615e8916e84 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.012301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.012910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.013194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.013471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.014236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] RBAC: Adding query filter params (interface=public, service_id=cdda209f65a14f06ab5228fe0eeb4a21, region_id=RegionOne) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:35.014550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.016194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.022232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 392/781] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/endpoints?service_id=cdda209f65a14f06ab5228fe0eeb4a21&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:35.029336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.064602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.124766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.147718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d09a45-d100-45dc-b601-8a8004155a23 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.169335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a41c90-7499-49e1-94b4-7c5d0e98b65f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.196748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059609e3-8265-4174-a193-d33e2c209c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.197814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059609e3-8265-4174-a193-d33e2c209c9d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.198198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059609e3-8265-4174-a193-d33e2c209c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.198577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059609e3-8265-4174-a193-d33e2c209c9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.208199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.209103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.209562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.209996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.210804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.211760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059609e3-8265-4174-a193-d33e2c209c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:35.212564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-059609e3-8265-4174-a193-d33e2c209c9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:35.218017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:35.219294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:35.220258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 390/782] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:35.221928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 393/783] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:35.222259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7e6b2-56b4-4797-b5b1-abfada8cf1ff None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.222612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7e6b2-56b4-4797-b5b1-abfada8cf1ff None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.222999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7e6b2-56b4-4797-b5b1-abfada8cf1ff None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.228592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 391/784] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:35.229998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.232175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b587ca-1e60-4ac1-a2ab-790597246535 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.232756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b587ca-1e60-4ac1-a2ab-790597246535 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.233636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b587ca-1e60-4ac1-a2ab-790597246535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.257993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-20b587ca-1e60-4ac1-a2ab-790597246535 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:35.259206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-20b587ca-1e60-4ac1-a2ab-790597246535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:35.393789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 392/785] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:35.396840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397a7b-f4b6-4b30-a024-880643fc1df5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.416436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b587ca-1e60-4ac1-a2ab-790597246535 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.431991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.451065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.451690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.451974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.452251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.453009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:35.453891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.454712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.465650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 394/786] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:35.470966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.471438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.557162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b587ca-1e60-4ac1-a2ab-790597246535 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.584368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a52840-1430-4ae0-ac0d-e578aa9a5521 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.606588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.607186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.607466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.607740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.621968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.622634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.622916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.623191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.629940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:35.634418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:35.635638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:35.636581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:35.637404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:35.638017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.639867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 393/787] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:35.653783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:35.659869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:35.660944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:35.662049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:35.662049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:35.662049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:35.662049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:35.664444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 395/788] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:35.669726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.741222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.846336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24878-db30-48f7-9e7f-578b7d4e8787 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.855215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.871793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059efc17-a9e3-4888-a193-faac1f655796 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.890292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.890921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.891203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.891525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.892187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:35.892535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.893949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.906683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 396/789] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:35.912357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.927671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.928284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.928598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.928874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.929598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:35.929904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.931279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.953589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 394/790] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:35.969652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.182072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.191118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a582-2dce-46d2-94e2-8ffc3e8c5037 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.198526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee0446-4680-4486-9d4f-7afe7cb22975 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.337490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.338162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.339199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.339199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.342259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.342259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.342259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.342259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.368418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.372406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.383635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.385117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:36.414226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.415478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:36.416432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:36.425461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 397/791] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 513 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:36.433446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:36.440183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.441141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.442196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:36.442196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:36.442196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:36.442196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:36.442196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 395/792] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:36.449528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.476710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.648569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4065-1b86-441e-9917-9295f3a8e81a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.654356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016c4dd-5cd9-4968-9de0-123517138e14 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.693172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bce23-ee88-4e7b-937d-58ecc115eae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.693778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bce23-ee88-4e7b-937d-58ecc115eae9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.694055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bce23-ee88-4e7b-937d-58ecc115eae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.694367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bce23-ee88-4e7b-937d-58ecc115eae9 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.695122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bce23-ee88-4e7b-937d-58ecc115eae9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:36.695428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bce23-ee88-4e7b-937d-58ecc115eae9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.695525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d77c41-9cc1-42b9-9e50-281d5e068980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.696112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d77c41-9cc1-42b9-9e50-281d5e068980 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.696455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d77c41-9cc1-42b9-9e50-281d5e068980 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.696689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d77c41-9cc1-42b9-9e50-281d5e068980 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.696891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bce23-ee88-4e7b-937d-58ecc115eae9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.697380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d77c41-9cc1-42b9-9e50-281d5e068980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:36.697684 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d77c41-9cc1-42b9-9e50-281d5e068980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.699027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d77c41-9cc1-42b9-9e50-281d5e068980 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.711903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 396/793] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:36.713594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f83de-be4e-4bac-97a3-bc36bdcf02c1 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.713882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f83de-be4e-4bac-97a3-bc36bdcf02c1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.714157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f83de-be4e-4bac-97a3-bc36bdcf02c1 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.719711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 397/794] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:36.719711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aeb54-04c6-4a5c-ad10-9746144fe4c7 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.719711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aeb54-04c6-4a5c-ad10-9746144fe4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.719711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71aeb54-04c6-4a5c-ad10-9746144fe4c7 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.719711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 398/795] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:36.720295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71aeb54-04c6-4a5c-ad10-9746144fe4c7 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.730563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 398/796] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:36.730563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6647076-c270-464e-ab83-26f47a403ea3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.730563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6647076-c270-464e-ab83-26f47a403ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.730563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6647076-c270-464e-ab83-26f47a403ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.748820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c6647076-c270-464e-ab83-26f47a403ea3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:36.750009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c6647076-c270-464e-ab83-26f47a403ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:36.761434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.793991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.824082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 399/797] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:36.828769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71aeb54-04c6-4a5c-ad10-9746144fe4c7 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.829843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006869b-edd2-4116-8fee-3280287ce9c5 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.830254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006869b-edd2-4116-8fee-3280287ce9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.831386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006869b-edd2-4116-8fee-3280287ce9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.854621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9006869b-edd2-4116-8fee-3280287ce9c5 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:36.856513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9006869b-edd2-4116-8fee-3280287ce9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:36.865931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.866581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.866865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.867143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.874540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.875968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:36.887903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.888971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:36.889871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 399/798] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:36.890442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.932666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 400/799] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:36.933502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006869b-edd2-4116-8fee-3280287ce9c5 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.005611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e259c70-cdac-4b0c-906b-a31c7b8da186 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.024114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.033476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.041834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.042533 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.042811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.043081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.048741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006869b-edd2-4116-8fee-3280287ce9c5 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.053515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.054950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:37.070186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.071086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.071966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:37.071966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:37.071966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.071966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:37.075904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 400/800] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.076750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.088028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.088994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.089518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.089974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.090900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.092471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.102177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.103144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:37.103942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:37.110945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 401/801] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.112512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.185005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ac542-4f58-4881-9cb8-9090aa6b3525 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.223681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.224302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.224619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.224894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.225647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c99e7-08ba-414a-91fc-1d37ff2c66a6 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.226094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.226094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.227386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.248568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 401/802] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.250799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.263301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.263853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.264137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.264432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.272690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.276574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:37.281466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.282456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:37.283329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 402/803] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.290192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.295812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.372806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbd6ac-0e4d-4bdc-842f-2674eb7bc74a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.394986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086a68b-4546-4265-9e93-cccbea066319 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.407656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.408569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.409607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.410097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.410965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.411447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.413135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.426163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 402/804] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.431522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.432456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.433072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.433354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.433632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.434320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.434630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.436004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.440699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 403/805] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.441476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.505130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.537899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.542278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a5c48-dfd8-47b9-bf65-69656f111df0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.548053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c81332-a69d-4efe-a09c-63928648a447 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.576397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.577056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.577340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.577617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.584479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.585098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.585468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.585743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.586450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.586749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.588101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.588973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.590391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.596069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 404/806] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.597864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:37.605041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.606094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.607131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:37.607131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:37.607131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.607131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:37.607131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 403/807] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.607455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.699600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60c320-3c91-4acd-9082-3151dc50a522 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.718175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc68174-062e-45e6-a590-915c85f88039 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.725426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.734328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bddd4-8b19-45f4-b0d5-163e54af5507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.734955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bddd4-8b19-45f4-b0d5-163e54af5507 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.735245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bddd4-8b19-45f4-b0d5-163e54af5507 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.735518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bddd4-8b19-45f4-b0d5-163e54af5507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.742328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bddd4-8b19-45f4-b0d5-163e54af5507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.743764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bddd4-8b19-45f4-b0d5-163e54af5507 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.751487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.752099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.752502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.752777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.753451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.753757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-b27bddd4-8b19-45f4-b0d5-163e54af5507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:37.754986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.755971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:37.757332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:37.757891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.757974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 405/808] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.757974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa9ff7-7bac-45d9-ad98-b6cea9a38e01 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.757974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa9ff7-7bac-45d9-ad98-b6cea9a38e01 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.757974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa9ff7-7bac-45d9-ad98-b6cea9a38e01 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.759179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 406/809] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:37.759919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa9ff7-7bac-45d9-ad98-b6cea9a38e01 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.766567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 404/810] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.767376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.783046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.863021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa9ff7-7bac-45d9-ad98-b6cea9a38e01 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.865199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2572973-2148-472b-81c6-5e56ace39448 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.897342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9c69-ab11-456f-8fef-999bfaacf9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.897756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.897978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9c69-ab11-456f-8fef-999bfaacf9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.898260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9c69-ab11-456f-8fef-999bfaacf9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.898350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.898537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9c69-ab11-456f-8fef-999bfaacf9c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.898627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.898872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.899550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.899848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.901232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.908874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9c69-ab11-456f-8fef-999bfaacf9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.910298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9c69-ab11-456f-8fef-999bfaacf9c6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.912389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 405/811] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.913177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-31ab9c69-ab11-456f-8fef-999bfaacf9c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:37.929198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.930285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.931351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:37.931351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:37.931351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.931351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:37.931351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 407/812] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.932247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6766-d94c-44c1-a695-d0fb41cc2669 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.932561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6766-d94c-44c1-a695-d0fb41cc2669 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.932835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b6766-d94c-44c1-a695-d0fb41cc2669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.950716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-226b6766-d94c-44c1-a695-d0fb41cc2669 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:37.951912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-226b6766-d94c-44c1-a695-d0fb41cc2669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:37.981395 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.991683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.015656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268e490-461c-44fa-ba75-80f2f25b56b0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.018014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 408/813] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:38.019092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6766-d94c-44c1-a695-d0fb41cc2669 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.053312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.053932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.054208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.054480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.055222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:38.055523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.056965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.068741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 406/814] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:38.069531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.117239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b6766-d94c-44c1-a695-d0fb41cc2669 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.153108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.153727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.154010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.154285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.154996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.156407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.173601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7807de5-5bb7-4019-b9b3-be24cd00c867 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.206970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.207599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.207877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.208149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.210019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.215429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.217893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.229814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 409/815] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:55:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:38.231778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:38.232053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:38.232955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:38.233885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 407/816] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:38.234367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.266564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.359387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fbf98-433f-47f4-bd67-c1ef5e83fd36 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.365808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb9fc8-54d1-48b5-b80f-e00b97a1a81e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.400815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.401777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.401777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.401948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.402530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.403900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.407570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.408303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.408742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.409117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.409920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:38.410234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.411590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:38.414865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:38.415814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:38.416713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:38.417443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 408/817] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:38.417998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.425896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 410/818] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:38.426598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.491876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.511207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.530010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457ff2-af6a-42a3-a741-fef934834ebe None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.538535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8dbf3-8928-402f-a740-f34f6feb11f5 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.574480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.575137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.575377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.575648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.576418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:38.576726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.578047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.578182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.578714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.578993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.579268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/dcebfbf0b3084929bc33887467695b00/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.581690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=dcebfbf0b3084929bc33887467695b00, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.583327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.591170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 411/819] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:38.591170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.641419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 409/820] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:38 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/dcebfbf0b3084929bc33887467695b00/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:38.642590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.724527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e860f82-9c61-4122-b594-6d538ba9a16f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.765306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.765944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.766226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.766501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.767289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:38.767590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.769636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.775222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 412/821] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:38.776121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.780621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.786473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e946f3c-2b41-4d64-a02e-209225bd657f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.807487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.828423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.829024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.829306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.829900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.839767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.841736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:38.856577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:38.857534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 410/822] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:38.859054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.900691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa44c8-9ae8-474d-b64b-317ff58793b5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.939425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.941050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.941545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.942061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/f247bde98344474ea5a62b4fa04464b8/roles/4981f801f6b14138984b3ae833a5797a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.958021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=f247bde98344474ea5a62b4fa04464b8, role_id=4981f801f6b14138984b3ae833a5797a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.959801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.990210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910b101-c5dc-4b26-937e-f6b037818ac8 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.026704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.027569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.027658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.027898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.028196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 413/823] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:38 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/f247bde98344474ea5a62b4fa04464b8/roles/4981f801f6b14138984b3ae833a5797a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:39.028664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] RBAC: Adding query filter params (type=network) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.028968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.029836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.030573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.037432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 411/824] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.042954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.120215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.155527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e150d-bf31-40cb-8d17-18ccb82a16f8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.166862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.205677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.206725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.207227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.207780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.208847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.209386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.211769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.233996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 414/825] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.240318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.270355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86cf3d7-bf33-4f5c-8303-3f144d85d798 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.313317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.314356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.315419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.315918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.316913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.318502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.365187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8206202-f7c3-4d9e-9436-ac69809989bd None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.382204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.407987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 412/826] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:55:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:39.421805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.422806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.423271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.423712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.503800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.505532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.620098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 415/827] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:55:39.625568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.748129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.752777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1307f-909b-464b-9e39-4694953834fb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.790086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.790664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.790982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.791363 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.803025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.804502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:39.819704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:39.821848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:39.822750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:39.822750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:39.822750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:39.822750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:39.822750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 413/828] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:39.828660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.840087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.943800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95963f6-745c-4e31-9899-c88957637217 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.986250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.986917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.987200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.987546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.988262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.990971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.992416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.005675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 416/829] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:40.013473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.121611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5254053-d2c7-4d3e-8c7b-78e3efec3df0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.155868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.156534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.156812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.157081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.166902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.168347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:40.179742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:40.180624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:40.181505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 414/830] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:40.188520 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.258753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.297947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3d45a-feeb-4e7a-9e99-9758fec3d33e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.332940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5d218-76b1-4452-b6d8-706d38f4de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.333563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5d218-76b1-4452-b6d8-706d38f4de26 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.333841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5d218-76b1-4452-b6d8-706d38f4de26 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.334112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5d218-76b1-4452-b6d8-706d38f4de26 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.334873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5d218-76b1-4452-b6d8-706d38f4de26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.335176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5d218-76b1-4452-b6d8-706d38f4de26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.336630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5d218-76b1-4452-b6d8-706d38f4de26 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.349560 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 417/831] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.357519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.370863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.464003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efba8c-7a6e-44d2-b3e9-7d655b3a2619 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.488757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff19311-1559-47a7-8f58-6711bf60630b None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.489061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff19311-1559-47a7-8f58-6711bf60630b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.489312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff19311-1559-47a7-8f58-6711bf60630b None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.490916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 418/832] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:40.495686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aef914-41ab-47a0-9164-58eefbaf8ae3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.495976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aef914-41ab-47a0-9164-58eefbaf8ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.496255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aef914-41ab-47a0-9164-58eefbaf8ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.499712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.500745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.501229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.501701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.503413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=784880e276bf4d64ae785609a463403a) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.503939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.505679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.512039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 415/833] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/role_assignments?user.id=784880e276bf4d64ae785609a463403a&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.516495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b6aef914-41ab-47a0-9164-58eefbaf8ae3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:40.516495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b6aef914-41ab-47a0-9164-58eefbaf8ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:40.596550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 419/834] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 27 19:55:40.618626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.703760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.728777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56d5f2-f32b-4a38-9786-0f519d5fb4f4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.765083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.766037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.766496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.766967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.767779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.769317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:40.778760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:40.779941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:40.780999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:40.783462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 416/835] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:40.789707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aef914-41ab-47a0-9164-58eefbaf8ae3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.856198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.899196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aef914-41ab-47a0-9164-58eefbaf8ae3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.935329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb317d4-bdca-4f63-b97b-9801da7bc1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.936225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb317d4-bdca-4f63-b97b-9801da7bc1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.936668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb317d4-bdca-4f63-b97b-9801da7bc1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.937088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb317d4-bdca-4f63-b97b-9801da7bc1e2 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.937930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb317d4-bdca-4f63-b97b-9801da7bc1e2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.938387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb317d4-bdca-4f63-b97b-9801da7bc1e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.939941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb317d4-bdca-4f63-b97b-9801da7bc1e2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.945276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 420/836] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.956422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.987053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be0533-b8de-4638-b9fb-564e6c5c024f None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.987350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be0533-b8de-4638-b9fb-564e6c5c024f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.987515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be0533-b8de-4638-b9fb-564e6c5c024f None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.992491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 421/837] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:40.994067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a777c22-2ff5-4389-89f6-da45e5d1e3b7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.994354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a777c22-2ff5-4389-89f6-da45e5d1e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.994629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a777c22-2ff5-4389-89f6-da45e5d1e3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.012667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6a777c22-2ff5-4389-89f6-da45e5d1e3b7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:41.013855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6a777c22-2ff5-4389-89f6-da45e5d1e3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:41.073017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89e5fb-b575-4775-8cfd-7bcc395d40c0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.098906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 422/838] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:41.116410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.117097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.117376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.117650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.118385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:41.118690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.120173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.122319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a777c22-2ff5-4389-89f6-da45e5d1e3b7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.131856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 417/839] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:41.189474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.246280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a777c22-2ff5-4389-89f6-da45e5d1e3b7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.285569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93664a3-1dcb-4d36-a926-7ad4efb6fe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.286307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93664a3-1dcb-4d36-a926-7ad4efb6fe84 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.286492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93664a3-1dcb-4d36-a926-7ad4efb6fe84 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.286767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93664a3-1dcb-4d36-a926-7ad4efb6fe84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.297719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93664a3-1dcb-4d36-a926-7ad4efb6fe84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-b93664a3-1dcb-4d36-a926-7ad4efb6fe84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:41.302711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:41.303633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:41.304485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 423/840] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:41.309867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.313409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.328224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ac554-260c-4131-b485-b6582d7ba2d1 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.330449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ac554-260c-4131-b485-b6582d7ba2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.330449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ac554-260c-4131-b485-b6582d7ba2d1 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.330916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 424/841] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:41.336212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe587e-209e-418f-bf66-0e8bbc53bbbd None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.336806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe587e-209e-418f-bf66-0e8bbc53bbbd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.337293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe587e-209e-418f-bf66-0e8bbc53bbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.357998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ddbe587e-209e-418f-bf66-0e8bbc53bbbd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:41.359517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ddbe587e-209e-418f-bf66-0e8bbc53bbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:41.435167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5ef1d-bbe8-46d4-bb60-a62497e5c299 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.440172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 425/842] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:41.463010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe587e-209e-418f-bf66-0e8bbc53bbbd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.476083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.477144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.477634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.478111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.479130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:41.479785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.481463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.491451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 418/843] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:41.496685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.515661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.568370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.576137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe587e-209e-418f-bf66-0e8bbc53bbbd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.611415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa8390-9218-4efe-8121-e2d66dee6ae7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.613055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.613827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.614207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.615285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.625731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:41.633007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:41.633834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:41.638824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 426/844] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:41.640082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.653701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.654686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.655174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.655670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.668820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.670703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:41.686565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:41.687493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:41.688367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:41.688367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:41.688367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:41.688367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:41.691159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 419/845] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:41.697653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.762894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce701ec6-33e4-4fd0-9c2c-39942689adf3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.765131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.806681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.807826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.808262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.808899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.810084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:41.810546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.812069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.824370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 427/846] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:41.826865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9785-6be7-483e-9eab-0d4bcc95f00b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.829276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.847604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.867827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.868872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.869390 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.869926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.870822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:41.871328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.873039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.887743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 420/847] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:41.893775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.938256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad10f1-eef3-4a0a-b501-c88345d2d6d3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.974421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.975112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.975394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.975751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.987824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.989238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:42.007883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:42.008803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:42.009676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:42.009676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:42.009676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:42.009676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:42.012518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 428/848] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:42.019141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.027618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436431-49fc-485f-a685-e02b2ca6cce2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.043025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.066413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.067109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.067395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.067671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.075376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.077124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:42.090727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:42.091892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:42.093039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:42.095347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 421/849] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:42.100787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.138994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.153428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266c485-7840-4bde-8ee7-1333da2b7f3e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.199691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.200841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.201307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.201788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.202673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:42.203191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.205169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.214359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd9584-0dfb-492b-be95-775ff252c761 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.219545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 429/850] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:42.224874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.258590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.259561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.259853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.260167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.261267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:42.261568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.263809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.292469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 422/851] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:42.293767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.327417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.358962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636314dd-41ed-40fc-8b6e-c15ed188a517 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.408430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.409459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.409954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.410957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.417422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.423926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.425169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aa8bc-9064-43bc-8c11-b9550b14857e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.431688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:42.442026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:42.442946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:42.444266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:42.446886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 430/852] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:42.453420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.467543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.468214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.468565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.468843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.541769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.543324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.559282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198b195-0bed-4995-a720-a6ebbf85e99d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.594147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.595647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.596141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.596645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.597643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:42.598146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.599767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.602693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.621617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 431/853] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:42.626958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.640619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.653259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 423/854] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:55:42.654940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.744462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa7e1f-a64c-4f9e-bf0f-76ca34c8b925 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.760078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138cf78-b3e2-429a-86aa-3b78423ca3ec None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.781165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14d084-c309-434b-aa31-80a495dba12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.782083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d084-c309-434b-aa31-80a495dba12c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.783211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d084-c309-434b-aa31-80a495dba12c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.783701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d084-c309-434b-aa31-80a495dba12c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.794304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03106af-2875-4551-b4af-f913732feb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.794881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03106af-2875-4551-b4af-f913732feb02 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.795161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03106af-2875-4551-b4af-f913732feb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.795435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03106af-2875-4551-b4af-f913732feb02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.808230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03106af-2875-4551-b4af-f913732feb02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.809724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03106af-2875-4551-b4af-f913732feb02 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-c03106af-2875-4551-b4af-f913732feb02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:42.825147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:42.826186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:42.827132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:42.827132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:42.827132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:42.827132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:42.827132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 424/855] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:42.827504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795dfcf-b261-44e0-8a92-f19bf03cbea9 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.827739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795dfcf-b261-44e0-8a92-f19bf03cbea9 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.828011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795dfcf-b261-44e0-8a92-f19bf03cbea9 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.829586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 425/856] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:42.833870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795dfcf-b261-44e0-8a92-f19bf03cbea9 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.856218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.864585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14d084-c309-434b-aa31-80a495dba12c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.866043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14d084-c309-434b-aa31-80a495dba12c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.886175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.945569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795dfcf-b261-44e0-8a92-f19bf03cbea9 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.972994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 432/857] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:55:42.972994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7153-d673-4158-ad0f-6dde3668cea7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.972994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7153-d673-4158-ad0f-6dde3668cea7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.972994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7153-d673-4158-ad0f-6dde3668cea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.988339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.988981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.989258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.989528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.990228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:42.990517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.991423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b77d7153-d673-4158-ad0f-6dde3668cea7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:42.991862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.993089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b77d7153-d673-4158-ad0f-6dde3668cea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:43.004317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 426/858] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:43.005125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.071265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 433/859] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:43.072416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7153-d673-4158-ad0f-6dde3668cea7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.110244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.113326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1e818-49f1-4d5a-80f5-17dfcdbaa5b4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.121950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.148940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.149594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.149879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.150159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.161724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.163167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:43.182010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:43.182917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:43.183778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:43.183778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:43.183778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:43.183778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:43.183778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 427/860] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:43.184046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.197552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7153-d673-4158-ad0f-6dde3668cea7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.235036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.236218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.236534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.236813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.243461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.244958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:43.256118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:43.257060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:43.257924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 434/861] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:43.258404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.285223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b257edc-07d4-483e-b0b3-4c10a2d14f13 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.318444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6f874-dd90-40d7-a3e8-5eeeb50028c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.319080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6f874-dd90-40d7-a3e8-5eeeb50028c0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.319460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6f874-dd90-40d7-a3e8-5eeeb50028c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.319634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6f874-dd90-40d7-a3e8-5eeeb50028c0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.320264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6f874-dd90-40d7-a3e8-5eeeb50028c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.321708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6f874-dd90-40d7-a3e8-5eeeb50028c0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-fbc6f874-dd90-40d7-a3e8-5eeeb50028c0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:43.331823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:43.333206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:43.334217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 428/862] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:43.335392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12f0a3-3ad1-4fc5-84d2-945d36b4e60d None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.335392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12f0a3-3ad1-4fc5-84d2-945d36b4e60d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.335392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12f0a3-3ad1-4fc5-84d2-945d36b4e60d None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.335392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 429/863] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:43.336456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12f0a3-3ad1-4fc5-84d2-945d36b4e60d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.352809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.362952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.368813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f3b9b-6d91-41fb-8a1a-801b16bffa54 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.406030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.406641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.406925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.407201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.408454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.408763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.410126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.421315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 435/864] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.422097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.439606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12f0a3-3ad1-4fc5-84d2-945d36b4e60d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.473829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a2b3a-e7e6-43dc-b52d-3fd387c56b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.474437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a2b3a-e7e6-43dc-b52d-3fd387c56b61 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.474718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a2b3a-e7e6-43dc-b52d-3fd387c56b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.474994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a2b3a-e7e6-43dc-b52d-3fd387c56b61 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.475737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a2b3a-e7e6-43dc-b52d-3fd387c56b61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.476045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a2b3a-e7e6-43dc-b52d-3fd387c56b61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.477500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a2b3a-e7e6-43dc-b52d-3fd387c56b61 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.488834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 430/865] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.489974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901a23e-b955-4b8b-b009-3ae758d3c3ae None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.490262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901a23e-b955-4b8b-b009-3ae758d3c3ae None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.490541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901a23e-b955-4b8b-b009-3ae758d3c3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.507452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3901a23e-b955-4b8b-b009-3ae758d3c3ae None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:43.508673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3901a23e-b955-4b8b-b009-3ae758d3c3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:43.523721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89a3ed-3859-4016-9a5b-bc65426e6714 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.556643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.557258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.557539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.557814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.558497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.558799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.560157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.564498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 436/866] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.565262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.573899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 431/867] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:43.574658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901a23e-b955-4b8b-b009-3ae758d3c3ae None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.576006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.589915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.665477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c1a5a-011a-4137-992b-d4c53eaac636 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.676740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901a23e-b955-4b8b-b009-3ae758d3c3ae None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.697733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.698372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.698655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.698940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.706013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.707443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.710541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.711215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.711560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.711900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.712776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=dcebfbf0b3084929bc33887467695b00) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.713157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.714618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:43.718667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:43.719657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:43.721445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 437/868] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:43.722135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 432/869] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/role_assignments?user.id=dcebfbf0b3084929bc33887467695b00&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.722135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.725329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.799037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.820434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.835660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf35574-84ba-4e4b-80ae-0abe4230e878 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.847254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99058a1e-f7c2-4346-b59c-087c62141ac3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.873937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.874566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.874842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.875112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.875804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.876637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.878005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.885008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 438/870] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.885785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.889451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.890106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.890392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.890668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.892278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.893510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.963548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 433/871] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:43 2022] POST /identity/v3/services => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:44.022123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd95a2-80e7-4bc5-97c6-1ccfdaf15ae0 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:44.056568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:44.070373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce1988-2135-4b0e-a0b5-966868b703c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:44.071016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce1988-2135-4b0e-a0b5-966868b703c1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:44.071295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce1988-2135-4b0e-a0b5-966868b703c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:44.071563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce1988-2135-4b0e-a0b5-966868b703c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:44.072348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce1988-2135-4b0e-a0b5-966868b703c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:44.072792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce1988-2135-4b0e-a0b5-966868b703c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:44.074368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce1988-2135-4b0e-a0b5-966868b703c1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:44.089853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 439/872] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:44.095567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:44.207150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450859a6-7b2f-4d5b-bcd8-8d15f77037dc None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:44.242503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34a9e6-ed8e-4f5d-9f3f-720e086aee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:44.243132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34a9e6-ed8e-4f5d-9f3f-720e086aee8c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:44.243416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34a9e6-ed8e-4f5d-9f3f-720e086aee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:44.243689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34a9e6-ed8e-4f5d-9f3f-720e086aee8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:44.244516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34a9e6-ed8e-4f5d-9f3f-720e086aee8c None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=f247bde98344474ea5a62b4fa04464b8) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:44.244829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34a9e6-ed8e-4f5d-9f3f-720e086aee8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:44.246265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34a9e6-ed8e-4f5d-9f3f-720e086aee8c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:44.249378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:44.253912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 434/873] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:44 2022] GET /identity/v3/role_assignments?user.id=f247bde98344474ea5a62b4fa04464b8&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:45.804038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70e19-d5e5-4842-853e-30f3bcdbb4f3 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.805138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70e19-d5e5-4842-853e-30f3bcdbb4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.805631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa70e19-d5e5-4842-853e-30f3bcdbb4f3 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.807434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 440/874] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:45.812721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12055fc-19ef-4935-a1dc-fc568ffd12fa None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.813530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12055fc-19ef-4935-a1dc-fc568ffd12fa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.814091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12055fc-19ef-4935-a1dc-fc568ffd12fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.832966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c12055fc-19ef-4935-a1dc-fc568ffd12fa None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:45.834466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c12055fc-19ef-4935-a1dc-fc568ffd12fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:45.905263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 435/875] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:45.927384 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70e19-d5e5-4842-853e-30f3bcdbb4f3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:45.940629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea4a80-d74e-4adb-9f18-25dfc4df6741 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.940928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea4a80-d74e-4adb-9f18-25dfc4df6741 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.941210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea4a80-d74e-4adb-9f18-25dfc4df6741 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.943209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 436/876] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:45.949721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6606b-a325-4b3e-b040-6d2c15e1dd36 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.950007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6606b-a325-4b3e-b040-6d2c15e1dd36 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.950272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6606b-a325-4b3e-b040-6d2c15e1dd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.969214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a9d6606b-a325-4b3e-b040-6d2c15e1dd36 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:45.971020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a9d6606b-a325-4b3e-b040-6d2c15e1dd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:46.040226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa70e19-d5e5-4842-853e-30f3bcdbb4f3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.047720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 437/877] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:46.066381 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6606b-a325-4b3e-b040-6d2c15e1dd36 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.084125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70998b-3c02-47dd-98fb-a7e9684aff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.085038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70998b-3c02-47dd-98fb-a7e9684aff6c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.085460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70998b-3c02-47dd-98fb-a7e9684aff6c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.085897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70998b-3c02-47dd-98fb-a7e9684aff6c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.086683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70998b-3c02-47dd-98fb-a7e9684aff6c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.088194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70998b-3c02-47dd-98fb-a7e9684aff6c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-af70998b-3c02-47dd-98fb-a7e9684aff6c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:46.097539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:46.098740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:46.099791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:46.102191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 441/878] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:46.103638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22464c7f-fd7e-4247-bc7c-7055c479cc61 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.103921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22464c7f-fd7e-4247-bc7c-7055c479cc61 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.104195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22464c7f-fd7e-4247-bc7c-7055c479cc61 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.105623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 442/879] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:46.109939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22464c7f-fd7e-4247-bc7c-7055c479cc61 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.137147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.169149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.178374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6606b-a325-4b3e-b040-6d2c15e1dd36 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.217373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.217994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.218279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.218444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22464c7f-fd7e-4247-bc7c-7055c479cc61 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.218543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.219234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.221401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.252926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05786b25-9fa0-40e6-8a83-062b42dd0722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.253550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05786b25-9fa0-40e6-8a83-062b42dd0722 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.253831 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05786b25-9fa0-40e6-8a83-062b42dd0722 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.254102 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05786b25-9fa0-40e6-8a83-062b42dd0722 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.254788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05786b25-9fa0-40e6-8a83-062b42dd0722 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:46.255094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05786b25-9fa0-40e6-8a83-062b42dd0722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.256488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05786b25-9fa0-40e6-8a83-062b42dd0722 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.261469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 443/880] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:46.262601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9af7-8276-48c3-a79b-b314872564ee None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.262884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9af7-8276-48c3-a79b-b314872564ee None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.263157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9af7-8276-48c3-a79b-b314872564ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.281513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-af3f9af7-8276-48c3-a79b-b314872564ee None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:46.282731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-af3f9af7-8276-48c3-a79b-b314872564ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:46.332491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 438/881] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:46 2022] POST /identity/v3/services => generated 269 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:55:46.333353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.382563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 444/882] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:46.383942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50999-16df-43b3-9c6c-cc0fb735de6f None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.384235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50999-16df-43b3-9c6c-cc0fb735de6f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.384548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50999-16df-43b3-9c6c-cc0fb735de6f None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.388459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 445/883] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:46.392172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af61e1-8638-4c76-a464-6e1f280cd212 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.392685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af61e1-8638-4c76-a464-6e1f280cd212 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.393104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af61e1-8638-4c76-a464-6e1f280cd212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.414025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-99af61e1-8638-4c76-a464-6e1f280cd212 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:46.415394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-99af61e1-8638-4c76-a464-6e1f280cd212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:46.454692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.466900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86bee1-7a9e-4674-b8d3-b4f4c503452b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.488744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.490199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 446/884] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:46.491409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af61e1-8638-4c76-a464-6e1f280cd212 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.505394 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.506477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.506764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.507075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.507966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:46.508233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.509685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.520481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 439/885] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:46.522999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.634314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af61e1-8638-4c76-a464-6e1f280cd212 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.662105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4183f-10ee-4040-a310-6464344b40cf None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.678357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.679753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.680020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.680290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.680971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.683102 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:46.694598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:46.695596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:46.696464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 447/886] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:46.701803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.709323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.710022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.710304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.710580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.737411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:46.742844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:46.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:46.746497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:46.747310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 440/887] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:46.753111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.812083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.821501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8028a-e9ae-4220-a2c1-5c8bed889684 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.857996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.858707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.859056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.859504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.860847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:46.861221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.862762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.868466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 448/888] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:46.874880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.877596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e468f-a772-4adf-99b1-57d7bc4416ab None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.915857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.916567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.916848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.917125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.917848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:46.918156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.919641 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.934029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 441/889] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:46.935017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.004395 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.007472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00442ff4-51dd-4413-8b39-23dd5ff1da58 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.045076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.045790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.046143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.046489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.047268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.047647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.049372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.054923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 449/890] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:47.059269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc9ac2-d622-4821-ab37-c8da4816e9f1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.060691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.067713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.096752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.097668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.098090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.098504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.109601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.112118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:47.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:47.128552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:47.129422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:47.129422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:47.129422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:47.129422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:47.131521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 442/891] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:47.137031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.177630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a804bc0-34f4-42ef-8ab3-32e13d08cd0c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.214108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.215165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.215673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.216196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.217364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] RBAC: Adding query filter params (interface=public, service_id=3dd35e9fa8964809a2959d5639093236, region_id=RegionOne) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.217877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.219529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.228753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 450/892] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/endpoints?service_id=3dd35e9fa8964809a2959d5639093236&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:47.231225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.243584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58ab98-955f-4f73-888d-2163a1279611 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.261974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.285670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.286324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.286601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.286871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.287571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.287897 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.289555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.306580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 443/893] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:47.307893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.338238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.369820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b505f-e3ef-4e24-b353-be9f8435daa4 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.417718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.418333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.418612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.418874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.419530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.420924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.435788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda9d7e-0524-4aa6-b97f-b5958246a36b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.436881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 451/894] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 19:55:47.476065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.477133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.477661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.478334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.487977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.489853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:47.504330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:47.505483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:47.506420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:47.509293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 444/895] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:47.515335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.620212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.628711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fde7e3-b92a-4262-a8e0-4800e144a576 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.667741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.668366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.668864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.669150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.669955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.670262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.671690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.690583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 452/896] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:47.696146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.781140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.803779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afafb2a-d35f-405c-ac64-398a09246a33 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.837375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.838045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.838323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.838593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.906058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.908030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.991563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.013498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 445/897] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:55:48.019812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.120919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3aa05-318d-4f49-b670-dabcad7632de None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.160356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.161041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.161359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.161638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.172981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.174503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:48.189965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:48.190842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:48.192288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:48.192288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:48.192288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:48.192288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:48.192288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 453/898] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:48.196817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.300404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0086e-65e0-4575-8ad2-eb66b49d9713 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.340752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06601a87-0699-43e7-955c-bcc79e94b6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.344411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06601a87-0699-43e7-955c-bcc79e94b6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.344411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06601a87-0699-43e7-955c-bcc79e94b6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.344411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06601a87-0699-43e7-955c-bcc79e94b6e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.344411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06601a87-0699-43e7-955c-bcc79e94b6e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:48.344411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06601a87-0699-43e7-955c-bcc79e94b6e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.344411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06601a87-0699-43e7-955c-bcc79e94b6e8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.356264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 446/899] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:48.361934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.371862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.469499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f640fa3-d084-43df-9319-352aceeb1c3d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.504773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.505417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.505700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.505978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.512804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.514315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.517601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a85b3-bb1e-4d2c-a73d-ce9f3f912198 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.517889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a85b3-bb1e-4d2c-a73d-ce9f3f912198 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.518174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a85b3-bb1e-4d2c-a73d-ce9f3f912198 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.519763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 447/900] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:48.524619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89add2a4-9e5c-4731-8ff1-bfaa1a122802 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.524907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89add2a4-9e5c-4731-8ff1-bfaa1a122802 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.525186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89add2a4-9e5c-4731-8ff1-bfaa1a122802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:48.526683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:48.527509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:48.528552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 454/901] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:48.537354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.544417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-89add2a4-9e5c-4731-8ff1-bfaa1a122802 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:48.545668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-89add2a4-9e5c-4731-8ff1-bfaa1a122802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:48.579631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.619639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 448/902] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:48.620276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.646810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89add2a4-9e5c-4731-8ff1-bfaa1a122802 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.667050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a940fd-1952-441b-844e-3b74c674fe21 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.711357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223590c4-d2e5-4587-833b-a552df9ffd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.712369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223590c4-d2e5-4587-833b-a552df9ffd89 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.712878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223590c4-d2e5-4587-833b-a552df9ffd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.713355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223590c4-d2e5-4587-833b-a552df9ffd89 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.714660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223590c4-d2e5-4587-833b-a552df9ffd89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:48.715180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223590c4-d2e5-4587-833b-a552df9ffd89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.716855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223590c4-d2e5-4587-833b-a552df9ffd89 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.729507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 455/903] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:48.731021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8f9b6-21bf-4055-b23b-322d436d64e4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.731369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8f9b6-21bf-4055-b23b-322d436d64e4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.731713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8f9b6-21bf-4055-b23b-322d436d64e4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.733143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 456/904] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:48.735659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8f9b6-21bf-4055-b23b-322d436d64e4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.761546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89add2a4-9e5c-4731-8ff1-bfaa1a122802 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.798868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d3f22-c86e-4377-9792-45046441a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.799708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d3f22-c86e-4377-9792-45046441a438 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.800210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d3f22-c86e-4377-9792-45046441a438 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.800780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d3f22-c86e-4377-9792-45046441a438 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.801410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d3f22-c86e-4377-9792-45046441a438 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.802765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d3f22-c86e-4377-9792-45046441a438 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-388d3f22-c86e-4377-9792-45046441a438 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:48.812517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:48.813952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:48.814785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 449/905] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:48.816115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763cc3a-ecaa-4ad5-ba14-55633483e426 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.816115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763cc3a-ecaa-4ad5-ba14-55633483e426 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.816278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763cc3a-ecaa-4ad5-ba14-55633483e426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.834624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c763cc3a-ecaa-4ad5-ba14-55633483e426 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:48.835731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c763cc3a-ecaa-4ad5-ba14-55633483e426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:48.846769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8f9b6-21bf-4055-b23b-322d436d64e4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.862830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.870230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.881934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.882556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.882839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.883115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.883908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] RBAC: Adding query filter params (role.id=0ded57e2af4f44de838c73c41429336f, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=f247bde98344474ea5a62b4fa04464b8) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:48.884219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.885671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.891054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 457/906] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/role_assignments?user.id=f247bde98344474ea5a62b4fa04464b8&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=0ded57e2af4f44de838c73c41429336f => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:48.891818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.928662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 450/907] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:48.953499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763cc3a-ecaa-4ad5-ba14-55633483e426 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.033932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcb814-33e5-4729-a774-a48fd019af83 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.083463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763cc3a-ecaa-4ad5-ba14-55633483e426 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.090513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.091224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.091611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.091992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.093281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.094244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.096097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.103106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 458/908] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:49.108243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.122686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.123631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.124108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.124587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.125545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.127076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.166181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.190843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 451/909] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:48 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:49.231225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eed5cb-1d28-44ea-b8ff-d0d06253071b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.278505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.279532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.280040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.280564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.282138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.282677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.284307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.291210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 459/910] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:49.297552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.329139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.419973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c24843-6365-4ab3-bc3e-ac7e029a4ea3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.459921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12dadd-bf5b-42a4-9884-de4c5264d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.460936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12dadd-bf5b-42a4-9884-de4c5264d315 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.461532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12dadd-bf5b-42a4-9884-de4c5264d315 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.462006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12dadd-bf5b-42a4-9884-de4c5264d315 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.462956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12dadd-bf5b-42a4-9884-de4c5264d315 None admin] RBAC: Adding query filter params (interface=public, service_id=e92449979cd5493aaefd7639f173a82a, region_id=RegionOne) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.463473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12dadd-bf5b-42a4-9884-de4c5264d315 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.465218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12dadd-bf5b-42a4-9884-de4c5264d315 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.474088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 452/911] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/endpoints?service_id=e92449979cd5493aaefd7639f173a82a&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:49.479823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.541835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7492-24b9-47f5-a959-bed3ecce2ee0 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.542465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7492-24b9-47f5-a959-bed3ecce2ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.542958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7492-24b9-47f5-a959-bed3ecce2ee0 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.544759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 453/912] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:49.552579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f285-5187-4122-b159-06125d36500a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.553106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f285-5187-4122-b159-06125d36500a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.553597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2f285-5187-4122-b159-06125d36500a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.572956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-34b2f285-5187-4122-b159-06125d36500a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:49.574660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-34b2f285-5187-4122-b159-06125d36500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:49.588450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bf293-2521-477b-bb42-d263edbea5b8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.600259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.630239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.631669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.632452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.632949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.634064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.636571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.646132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 460/913] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:55:49.667421 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 454/914] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:49.692805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.808117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5474-813c-4df0-a36d-fdfbf0d3bab1 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.844859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.845851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.846327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.846819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.847666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.849243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:49.858545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:49.859521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:49.860329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:49.863079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 461/915] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:49.869212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2f285-5187-4122-b159-06125d36500a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.887586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.972138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2f285-5187-4122-b159-06125d36500a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:50.007644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:50.008273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.008588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.008863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.009544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:50.009847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:50.011216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:50.020600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 455/916] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:50.025478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.071388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:50.132996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86becbb7-6f6b-430b-8f27-74b435a32aed None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:50.169077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986c70b-f27c-4ea6-bb72-370be83c16fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:50.169694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986c70b-f27c-4ea6-bb72-370be83c16fa None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.169976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986c70b-f27c-4ea6-bb72-370be83c16fa None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.170251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986c70b-f27c-4ea6-bb72-370be83c16fa None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.170933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986c70b-f27c-4ea6-bb72-370be83c16fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:50.171237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986c70b-f27c-4ea6-bb72-370be83c16fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:50.173191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986c70b-f27c-4ea6-bb72-370be83c16fa None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:50.178579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 462/917] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:50.184238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.266901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:50.286002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014b6c-bbaa-4602-ad0d-5e19b7d82783 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:50.320280 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a5603-62a7-412c-885e-a75bb9b8b254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:50.320958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5603-62a7-412c-885e-a75bb9b8b254 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.321300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5603-62a7-412c-885e-a75bb9b8b254 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.321971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a5603-62a7-412c-885e-a75bb9b8b254 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.322698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a5603-62a7-412c-885e-a75bb9b8b254 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:50.324057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a5603-62a7-412c-885e-a75bb9b8b254 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:50.403479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 456/918] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:55:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:55:50.970801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddd364-4925-41d8-89c2-776fedf549b4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.971205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddd364-4925-41d8-89c2-776fedf549b4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.971392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddd364-4925-41d8-89c2-776fedf549b4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.973006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 463/919] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:50.978108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e0dd2-9a65-4718-bea8-b23c72733220 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.978543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e0dd2-9a65-4718-bea8-b23c72733220 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.978725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e0dd2-9a65-4718-bea8-b23c72733220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.997262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-dc7e0dd2-9a65-4718-bea8-b23c72733220 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:50.998452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-dc7e0dd2-9a65-4718-bea8-b23c72733220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:51.074281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 457/920] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:51.096544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddd364-4925-41d8-89c2-776fedf549b4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.205008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddd364-4925-41d8-89c2-776fedf549b4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.206706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.240532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffbfff-84ee-4a86-bafc-2c41ea6b2939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.241174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffbfff-84ee-4a86-bafc-2c41ea6b2939 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.241452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffbfff-84ee-4a86-bafc-2c41ea6b2939 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.241727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffbfff-84ee-4a86-bafc-2c41ea6b2939 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.252781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffbfff-84ee-4a86-bafc-2c41ea6b2939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-45ffbfff-84ee-4a86-bafc-2c41ea6b2939 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:51.257732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:51.258623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:51.260087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 464/921] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:51.265295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e0dd2-9a65-4718-bea8-b23c72733220 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.352788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.363287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99992e7e-ecdb-473a-b034-4240df7f9461 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.363581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99992e7e-ecdb-473a-b034-4240df7f9461 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.363865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99992e7e-ecdb-473a-b034-4240df7f9461 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.365500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 465/922] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:51.370511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e0dd2-9a65-4718-bea8-b23c72733220 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.370625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65908-e7ca-4da1-8de5-4d9dc0001956 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.370867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65908-e7ca-4da1-8de5-4d9dc0001956 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.371288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af65908-e7ca-4da1-8de5-4d9dc0001956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.390483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-5af65908-e7ca-4da1-8de5-4d9dc0001956 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:51.391784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-5af65908-e7ca-4da1-8de5-4d9dc0001956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:51.410402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.411687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.412695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.413175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.414249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.415024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.417021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.429184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 458/923] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:51.440402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.480479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 466/924] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:51.504121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af65908-e7ca-4da1-8de5-4d9dc0001956 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.561335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be247d-0a04-44fd-bede-a950241dd99b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.594558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.606026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.610819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.610819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.610819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.620014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.621615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.621829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.632412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af65908-e7ca-4da1-8de5-4d9dc0001956 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:51.641612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:51.642655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:51.643710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:51.643710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:51.643710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:51.643710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:51.648784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 459/925] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:51.650619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.684095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.684900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.685195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.685470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.686108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.687751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:51.697098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:51.698320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:51.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:51.703532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 467/926] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:51.704730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.773673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44770133-469b-4253-96ab-ba2552b0cae1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.812314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d68e0b-e41c-4248-9901-804df38d2c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.812618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4022c-afce-4781-8381-d11c723f808e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.813337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d68e0b-e41c-4248-9901-804df38d2c0f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.813769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d68e0b-e41c-4248-9901-804df38d2c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.814404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d68e0b-e41c-4248-9901-804df38d2c0f None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.816073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d68e0b-e41c-4248-9901-804df38d2c0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.816586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d68e0b-e41c-4248-9901-804df38d2c0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.818098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d68e0b-e41c-4248-9901-804df38d2c0f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.833185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 460/927] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:51.834252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d82a43-5e51-4227-a09b-17657b2fa532 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.834534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d82a43-5e51-4227-a09b-17657b2fa532 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.834805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d82a43-5e51-4227-a09b-17657b2fa532 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.836207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 461/928] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:51.840727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d82a43-5e51-4227-a09b-17657b2fa532 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.853141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5fd14-04e1-4d1e-a1c5-1a70e2a262c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.853762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5fd14-04e1-4d1e-a1c5-1a70e2a262c0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.854050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5fd14-04e1-4d1e-a1c5-1a70e2a262c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.854331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5fd14-04e1-4d1e-a1c5-1a70e2a262c0 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.855013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5fd14-04e1-4d1e-a1c5-1a70e2a262c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.855322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5fd14-04e1-4d1e-a1c5-1a70e2a262c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.856741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5fd14-04e1-4d1e-a1c5-1a70e2a262c0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.866015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 468/929] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:51.866015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06ca8-954b-498f-a02f-8e3e153839ae None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.866015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06ca8-954b-498f-a02f-8e3e153839ae None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.866015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b06ca8-954b-498f-a02f-8e3e153839ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.881306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-49b06ca8-954b-498f-a02f-8e3e153839ae None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:51.882566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-49b06ca8-954b-498f-a02f-8e3e153839ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:51.883900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.898130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.951270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d82a43-5e51-4227-a09b-17657b2fa532 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.954699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 469/930] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:51.957269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06ca8-954b-498f-a02f-8e3e153839ae None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.991257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.991949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.992247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.992483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.000516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.002082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:52.017335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:52.018291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 462/931] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:52.019236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.069952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b06ca8-954b-498f-a02f-8e3e153839ae None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.106509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.107177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.107424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.107694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.108399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.108739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.110111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.115075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 470/932] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.116522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.134918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.140899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.147111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b913b-a163-4cd8-a13d-932b84756a97 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.181504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.182267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.182756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.183111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.183751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.185163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:52.195462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:52.196480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:52.201722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 463/933] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:52.203403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.232404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef82fb-a191-4e95-98d4-e80d59891e5d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.269161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.270267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.270561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.270839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.271807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.272121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.273741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.292157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 471/934] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.292999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.308418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738f17a-0337-4062-b978-b744f8d13e5d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.351077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.351790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.352069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.352339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.353125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] RBAC: Adding query filter params (interface=public, service_id=f879100d23504b32b217b6bd51c9cade, region_id=RegionOne) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.353427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.354780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.360216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 464/935] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/endpoints?service_id=f879100d23504b32b217b6bd51c9cade&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.360892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.383380 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.389674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.428160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ff46a-1569-4a7a-9590-66cffbc18a76 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.465981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.466689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.467037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.467379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.468160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.468566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.469993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.473414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af3b3e-a8f3-4c91-9583-efdb6f1769ff None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.475039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 472/936] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.480134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.507849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.508597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.508876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.509147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.519808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.521262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:52.535101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:52.536030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:52.536942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:52.536942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:52.536942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:52.536942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:52.536942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 465/937] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:52.537481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.584097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260ecd0-1202-4948-98de-e23e6eb50756 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.611666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.618789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f0e06-dd45-4f4b-956c-2f382e213e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.619393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f0e06-dd45-4f4b-956c-2f382e213e44 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.619668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f0e06-dd45-4f4b-956c-2f382e213e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.622249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f0e06-dd45-4f4b-956c-2f382e213e44 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.622249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f0e06-dd45-4f4b-956c-2f382e213e44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.622529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f0e06-dd45-4f4b-956c-2f382e213e44 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.623987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.629025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 473/938] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:55:52.630217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c35f6-522f-4ea2-b185-2b5179afee34 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.630509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c35f6-522f-4ea2-b185-2b5179afee34 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.630765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c35f6-522f-4ea2-b185-2b5179afee34 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.631982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 474/939] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:52.632725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c35f6-522f-4ea2-b185-2b5179afee34 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.638905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd522a-4cba-49ef-bf67-7ec853bc215a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.680346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a5c85-6c3f-4ae8-a6de-ca7af3456b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.681022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a5c85-6c3f-4ae8-a6de-ca7af3456b5d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.681309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a5c85-6c3f-4ae8-a6de-ca7af3456b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.681587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a5c85-6c3f-4ae8-a6de-ca7af3456b5d None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.682290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a5c85-6c3f-4ae8-a6de-ca7af3456b5d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.682601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a5c85-6c3f-4ae8-a6de-ca7af3456b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.684023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a5c85-6c3f-4ae8-a6de-ca7af3456b5d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.695239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 466/940] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.695239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb50d2d-6eee-4e43-83be-30cb5b98a739 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.695239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb50d2d-6eee-4e43-83be-30cb5b98a739 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.695239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb50d2d-6eee-4e43-83be-30cb5b98a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.711299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-feb50d2d-6eee-4e43-83be-30cb5b98a739 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:52.712576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-feb50d2d-6eee-4e43-83be-30cb5b98a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:52.748916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c35f6-522f-4ea2-b185-2b5179afee34 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.787580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 467/941] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:52.788239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb50d2d-6eee-4e43-83be-30cb5b98a739 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.807737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.808715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.809001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.809275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.809980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.810426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.812801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.824899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 475/942] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.826159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.863003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.890067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.894470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb50d2d-6eee-4e43-83be-30cb5b98a739 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.930687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.931294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.936876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.936876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.937042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efa27b-77cd-48ee-8b1c-63dec0d69e13 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.937159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.937159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.965891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.966508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.966786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.967054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.968255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.969701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.972264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING keystone.server.flask.common [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 19:55:53.020546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.sql.core [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 19:55:53.020546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 19:55:53.020546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [parameters: {'id': 'faaf961ce9934252bb189694bc646ae3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 19:55:53.020546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80335) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id faaf961ce9934252bb189694bc646ae3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id faaf961ce9934252bb189694bc646ae3. Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:53.027647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 19:55:53.028620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application yield resource Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise self.value Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application session.commit() Jul 27 19:55:53.029621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.session.flush() Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application compat.raise_( Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 19:55:53.031085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 19:55:53.031961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:53.032790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [parameters: {'id': 'faaf961ce9934252bb189694bc646ae3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.033694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.034705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id faaf961ce9934252bb189694bc646ae3. Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 476/943] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Jul 27 19:55:52 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.036752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.050729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 468/944] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:55:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:53.052208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.177515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee216cb0-868e-4537-9c74-034f9e15f009 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.197448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.201738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108d962-0267-418e-b56c-b92815862d32 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.220620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.221675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.222160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.222951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.231329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.232863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.235267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:53.248437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.250075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:53.251297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 477/945] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.252119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.252819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.253102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.253674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.256466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.267020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.268516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:53.286016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.287190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:53.288058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:53.289873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 469/946] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.295577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.384770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea71d6-9a74-4b09-9bba-d884e757e2d3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.419808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc135bb-0846-480a-860c-dc7192567fa0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.420317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.421738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.422221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.422707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.423665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.424188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.425994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.448083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 478/947] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:53.449340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.457209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b74fda-d56e-4fb6-b425-40a6aaad70f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.457890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b74fda-d56e-4fb6-b425-40a6aaad70f4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.458227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b74fda-d56e-4fb6-b425-40a6aaad70f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.458505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b74fda-d56e-4fb6-b425-40a6aaad70f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.459240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b74fda-d56e-4fb6-b425-40a6aaad70f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.459547 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b74fda-d56e-4fb6-b425-40a6aaad70f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.461186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b74fda-d56e-4fb6-b425-40a6aaad70f4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.488873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.489240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 470/948] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:53.563946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e64ff-b597-4ca9-81a7-76ec9f1ca111 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.603795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f819b8-96dd-49c4-8d5f-93c45f2e6a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.604919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f819b8-96dd-49c4-8d5f-93c45f2e6a96 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.605406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f819b8-96dd-49c4-8d5f-93c45f2e6a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.605908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f819b8-96dd-49c4-8d5f-93c45f2e6a96 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/f247bde98344474ea5a62b4fa04464b8/roles/0ded57e2af4f44de838c73c41429336f` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.609396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f819b8-96dd-49c4-8d5f-93c45f2e6a96 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=f247bde98344474ea5a62b4fa04464b8, role_id=0ded57e2af4f44de838c73c41429336f)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.611174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f819b8-96dd-49c4-8d5f-93c45f2e6a96 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.667804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 479/949] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:53 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/f247bde98344474ea5a62b4fa04464b8/roles/0ded57e2af4f44de838c73c41429336f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:54.690538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15158599-a67f-4be0-b977-0f0363d56633 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.691535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15158599-a67f-4be0-b977-0f0363d56633 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.692044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15158599-a67f-4be0-b977-0f0363d56633 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.693853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 471/950] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:54.699616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f018-1928-43df-b729-91c5a7bbe1fc None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.700424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f018-1928-43df-b729-91c5a7bbe1fc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.700925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f018-1928-43df-b729-91c5a7bbe1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.721034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-e363f018-1928-43df-b729-91c5a7bbe1fc None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:54.722895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e363f018-1928-43df-b729-91c5a7bbe1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:54.795700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 480/951] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:54.818441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15158599-a67f-4be0-b977-0f0363d56633 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.866785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.930115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15158599-a67f-4be0-b977-0f0363d56633 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.965376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039233ad-3af8-4375-979f-813d5b85c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.965948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039233ad-3af8-4375-979f-813d5b85c2fb None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.966230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039233ad-3af8-4375-979f-813d5b85c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.966504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039233ad-3af8-4375-979f-813d5b85c2fb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.967133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039233ad-3af8-4375-979f-813d5b85c2fb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.969075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039233ad-3af8-4375-979f-813d5b85c2fb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-039233ad-3af8-4375-979f-813d5b85c2fb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:54.978127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:54.979360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:54.980243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 472/952] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:54.988168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363f018-1928-43df-b729-91c5a7bbe1fc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.017412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.092482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363f018-1928-43df-b729-91c5a7bbe1fc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.127490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.128120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.128421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.128705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.129399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.129705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.131079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.136586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 481/953] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.138268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4434c9-0f89-4825-ae0b-935955c8390a None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.138560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4434c9-0f89-4825-ae0b-935955c8390a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.138837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4434c9-0f89-4825-ae0b-935955c8390a None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.140989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 473/954] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:55.143231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.146007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2436e-04b5-42be-a454-7f484d1445c8 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.146291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2436e-04b5-42be-a454-7f484d1445c8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.146564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2436e-04b5-42be-a454-7f484d1445c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.165244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-24d2436e-04b5-42be-a454-7f484d1445c8 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:55.166572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-24d2436e-04b5-42be-a454-7f484d1445c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:55.248292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 474/955] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:55.260838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.266632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85495991-93ce-498a-993e-2a3af48070a8 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.275314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2436e-04b5-42be-a454-7f484d1445c8 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.302229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.302816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.303083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.303355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.304036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.304342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.306049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.312800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 482/956] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.314951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.318421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.396519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2436e-04b5-42be-a454-7f484d1445c8 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.429981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be54aa7-f1d5-4505-b9a2-1ca7cd2f7cd6 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.434810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.435466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.435747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.436592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.437384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.438821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:55.449141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:55.450140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:55:55.451049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:55.455403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 475/957] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:55.456315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.471161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730fdc8-bf2a-427e-b655-09b519f65eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.472048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730fdc8-bf2a-427e-b655-09b519f65eed None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.472557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730fdc8-bf2a-427e-b655-09b519f65eed None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.473030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730fdc8-bf2a-427e-b655-09b519f65eed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.473914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730fdc8-bf2a-427e-b655-09b519f65eed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.475473 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730fdc8-bf2a-427e-b655-09b519f65eed None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.552911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 483/958] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:55:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:55.568051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.571652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0828-5254-4ca4-916e-a8705d20aef7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.590207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b2d4-945f-4d7a-b412-afdaa7b2a2d3 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.590533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b2d4-945f-4d7a-b412-afdaa7b2a2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.591188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b2d4-945f-4d7a-b412-afdaa7b2a2d3 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.596242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 484/959] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:55.597984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221415a6-088a-4b9c-a374-72c4e556b18b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.598295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221415a6-088a-4b9c-a374-72c4e556b18b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.598549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221415a6-088a-4b9c-a374-72c4e556b18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.618992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-221415a6-088a-4b9c-a374-72c4e556b18b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:55.620199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-221415a6-088a-4b9c-a374-72c4e556b18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:55.620793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.621428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.621711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.621987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.623009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.624634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.625100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.626222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.633896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 476/960] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.640486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.713909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 485/961] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:55.733654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221415a6-088a-4b9c-a374-72c4e556b18b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.769256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55b21-c768-4932-9808-ad33676fd882 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.807316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba279b6-74a5-4773-b0fa-dcac7e4ca2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.807879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba279b6-74a5-4773-b0fa-dcac7e4ca2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.808157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba279b6-74a5-4773-b0fa-dcac7e4ca2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.808506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba279b6-74a5-4773-b0fa-dcac7e4ca2e4 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.809191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba279b6-74a5-4773-b0fa-dcac7e4ca2e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.809493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba279b6-74a5-4773-b0fa-dcac7e4ca2e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.810877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba279b6-74a5-4773-b0fa-dcac7e4ca2e4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.816560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 477/962] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.817856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb2796-38e4-49e5-b711-8df3718ef065 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.818149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb2796-38e4-49e5-b711-8df3718ef065 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.818424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb2796-38e4-49e5-b711-8df3718ef065 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.820431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 478/963] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:55.825523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141665fe-f020-49c6-8ad5-4885968cf669 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.826099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141665fe-f020-49c6-8ad5-4885968cf669 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.826658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141665fe-f020-49c6-8ad5-4885968cf669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.847101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-141665fe-f020-49c6-8ad5-4885968cf669 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:55.848720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-141665fe-f020-49c6-8ad5-4885968cf669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:55.861253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221415a6-088a-4b9c-a374-72c4e556b18b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.869879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.899665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.900310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.900620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.900895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.901586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.903039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.930379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 479/964] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:55.955031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141665fe-f020-49c6-8ad5-4885968cf669 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.969239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.sql.core [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 19:55:55.969239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 19:55:55.969239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [parameters: {'user_id': '392555a098d34f0f961f267c8158e06c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 19:55:55.969239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80335) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:55.977893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:55.978866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application yield resource Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise self.value Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application session.commit() Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 19:55:55.979818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.session.flush() Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application compat.raise_( Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 19:55:55.981934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:55.983127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:55.995038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [parameters: {'user_id': '392555a098d34f0f961f267c8158e06c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:55.997645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 19:55:55.999147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 486/965] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Jul 27 19:55:55 2022] POST /identity/v3/users => generated 160 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.000117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.092069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141665fe-f020-49c6-8ad5-4885968cf669 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.112742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49df97-7d60-46e1-a81d-e44d7d1af9ad None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.134437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.135614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.141742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.141742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.150413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:56.150930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.151680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.152077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.152478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:56.156303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:56.157527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:56.158388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 480/966] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:56.163619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.167127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.168860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.183681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:56.184146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:56.185107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:56.186648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:56.190681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 487/967] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:56.191565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.256308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.298753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427e2f7-56e0-4bf4-81d5-b330a840f327 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.304095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5233dc-19ea-4c18-97a2-19235e03b353 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.335113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.335705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.335989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.336262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.337120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:56.337319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.338694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.352519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 481/968] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:56.352669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.352786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.353170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.353274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.354038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:56.354341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.355117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.355873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.400707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 488/969] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:56.448831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.475026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd48e59-ecb0-4f56-8651-53da694fd8a0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.513317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.514446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.514928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.515411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.527581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.529469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:56.546095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:56.547035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:56.547915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:56.547915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:56.547915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:56.547915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:56.554483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 482/970] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:56.555444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.663838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206cb17-2a7b-48fa-ad05-6c46ff3bd22c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.685114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.699178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.714963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.714963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.714963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.714963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:56.714963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.714963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.715851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 489/971] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:56.720331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.830734 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92178f0f-d6ce-4364-ae6a-e332cd1a681a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.856784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.865408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.869625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.869625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.869625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.874231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.875722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:56.886743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:56.887964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:56.889247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:56.890335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 483/972] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:56.895864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.996573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49432e7f-33d8-4847-bafa-f3e22ce27d82 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.029179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.030333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.030612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.030894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.031667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:57.031969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.033417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.050327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 490/973] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:57.055861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.156498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5c182-cc9b-453b-b78e-0bb54e406c94 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.189038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.189774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.190159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.190526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.244693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.258147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.258571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.358929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 484/974] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:55:57.364269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.407614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.470898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aedfe0-f9f7-4b09-ba1e-791106c50858 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.504892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32457444-da58-4446-87d3-7e11e48600b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.505534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457444-da58-4446-87d3-7e11e48600b7 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.505817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457444-da58-4446-87d3-7e11e48600b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.506092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32457444-da58-4446-87d3-7e11e48600b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.516474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32457444-da58-4446-87d3-7e11e48600b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.517919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32457444-da58-4446-87d3-7e11e48600b7 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-32457444-da58-4446-87d3-7e11e48600b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:57.533168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:57.534106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:57.534997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:57.534997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:57.534997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:57.534997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:57.536709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 491/975] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:57.540166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.630845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.643572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac52bd5-5609-44a7-8ded-f2f9dc57502d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.676842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e409ce-d273-4b8f-aa05-52a074a7c271 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.677177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e409ce-d273-4b8f-aa05-52a074a7c271 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.677394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e409ce-d273-4b8f-aa05-52a074a7c271 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.678864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 492/976] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:57.681057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.681675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.681955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.682227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.682918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:57.683221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.683728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59af30-0941-4ccd-9304-40933fa4c42d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.684018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59af30-0941-4ccd-9304-40933fa4c42d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.684291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59af30-0941-4ccd-9304-40933fa4c42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.684605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.700804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 485/977] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:57.702189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.704085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ac59af30-0941-4ccd-9304-40933fa4c42d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:57.705639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ac59af30-0941-4ccd-9304-40933fa4c42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:57.781362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 493/978] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:57.803157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59af30-0941-4ccd-9304-40933fa4c42d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.810458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c995f3c-8f7b-4a60-9e4e-0a836394e903 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.830343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.847131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.847771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.848055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.848333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.855351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.856844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:57.868502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:57.869576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:55:57.870558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 486/979] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:57.875714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.887333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.913568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59af30-0941-4ccd-9304-40933fa4c42d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.946509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b9b6c-9645-4714-9e56-ca91cd1e2f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.947144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9b6c-9645-4714-9e56-ca91cd1e2f5a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.947426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9b6c-9645-4714-9e56-ca91cd1e2f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.947699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9b6c-9645-4714-9e56-ca91cd1e2f5a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.948330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b9b6c-9645-4714-9e56-ca91cd1e2f5a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.949717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b9b6c-9645-4714-9e56-ca91cd1e2f5a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-527b9b6c-9645-4714-9e56-ca91cd1e2f5a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:57.959330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:57.960355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:55:57.961514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 494/980] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:57.962371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2368483-42ed-42c0-a05e-c0bc0e982844 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.962371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2368483-42ed-42c0-a05e-c0bc0e982844 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.962371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2368483-42ed-42c0-a05e-c0bc0e982844 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.963412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 495/981] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:55:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:57.966528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2368483-42ed-42c0-a05e-c0bc0e982844 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.980705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ddcf-b402-4a9d-80e8-0443729ae5ff None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.012857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109707c-9190-4808-920d-4d3181d6100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.013968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109707c-9190-4808-920d-4d3181d6100c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.014238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109707c-9190-4808-920d-4d3181d6100c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.014504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109707c-9190-4808-920d-4d3181d6100c None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.015240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109707c-9190-4808-920d-4d3181d6100c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.015540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109707c-9190-4808-920d-4d3181d6100c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.016966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109707c-9190-4808-920d-4d3181d6100c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.028647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 487/982] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:58.030597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f7e6-7912-40e0-9426-d68f083cb0e8 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.031138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f7e6-7912-40e0-9426-d68f083cb0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.031616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3f7e6-7912-40e0-9426-d68f083cb0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.048981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-9ce3f7e6-7912-40e0-9426-d68f083cb0e8 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:58.050167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9ce3f7e6-7912-40e0-9426-d68f083cb0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:58.051469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.068668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2368483-42ed-42c0-a05e-c0bc0e982844 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.102960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.103569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.103849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.104122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.104839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.105142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.106503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.110902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.115748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 496/983] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:58.115748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.120891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 488/984] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:58.126062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f7e6-7912-40e0-9426-d68f083cb0e8 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.219663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3f7e6-7912-40e0-9426-d68f083cb0e8 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.220183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb800864-9203-45d5-ad0b-6d45a150a565 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.252510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.252929 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.253118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.253398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.253532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.253670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.253814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.254086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.254344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.254645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.254869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] RBAC: Adding query filter params (role.id=0ded57e2af4f44de838c73c41429336f, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=f247bde98344474ea5a62b4fa04464b8) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.255178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.255999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.256595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.260695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 489/985] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:58.261441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.270022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 497/986] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/role_assignments?user.id=f247bde98344474ea5a62b4fa04464b8&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=0ded57e2af4f44de838c73c41429336f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:58.270022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.288556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.346163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.390017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30180454-c86a-4f62-a945-a30800e67982 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.390521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96533f2-2a05-442c-869e-265d24ab0864 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.431071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8df46b-da31-4c67-9ccd-6c6ba7da99ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.431689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8df46b-da31-4c67-9ccd-6c6ba7da99ee None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.431968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8df46b-da31-4c67-9ccd-6c6ba7da99ee None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.432242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8df46b-da31-4c67-9ccd-6c6ba7da99ee None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.433007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8df46b-da31-4c67-9ccd-6c6ba7da99ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.435110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.435300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8df46b-da31-4c67-9ccd-6c6ba7da99ee None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.435387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.435387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.435387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.437248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] RBAC: Adding query filter params (interface=admin, service_id=f879100d23504b32b217b6bd51c9cade, region_id=RegionOne) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.437248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.438298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.444571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 498/987] 10.222.0.47 () {56 vars in 1249 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/endpoints?service_id=f879100d23504b32b217b6bd51c9cade&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:58.449300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.501561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 490/988] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:58 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:58.533048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.558273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10616b91-d255-421d-ab1f-83814af5d334 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.603061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b56fc-a620-454c-89a3-9864167e41c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.603683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b56fc-a620-454c-89a3-9864167e41c7 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.603960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b56fc-a620-454c-89a3-9864167e41c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.604229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b56fc-a620-454c-89a3-9864167e41c7 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.604934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b56fc-a620-454c-89a3-9864167e41c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.606318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b56fc-a620-454c-89a3-9864167e41c7 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.620447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 499/989] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 19:56:00.287025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53425aa3-0ecf-4ff6-8bb6-9f8fd85e843e None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.287577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53425aa3-0ecf-4ff6-8bb6-9f8fd85e843e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.287577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53425aa3-0ecf-4ff6-8bb6-9f8fd85e843e None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.289088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 491/990] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:00.294051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b136d-c765-4c12-a308-ecfb8746f4ab None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.294556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b136d-c765-4c12-a308-ecfb8746f4ab None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.294645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b136d-c765-4c12-a308-ecfb8746f4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.313837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-028b136d-c765-4c12-a308-ecfb8746f4ab None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:00.315050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-028b136d-c765-4c12-a308-ecfb8746f4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:00.391738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 500/991] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:00.413431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53425aa3-0ecf-4ff6-8bb6-9f8fd85e843e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.520957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53425aa3-0ecf-4ff6-8bb6-9f8fd85e843e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.537349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58494-7ff5-4223-8cc9-c417eec0ae35 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.537623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58494-7ff5-4223-8cc9-c417eec0ae35 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.537832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da58494-7ff5-4223-8cc9-c417eec0ae35 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.539330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 501/992] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:00.543918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708edaf0-471d-4dcd-9670-b6975a5807d6 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.544248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708edaf0-471d-4dcd-9670-b6975a5807d6 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.544552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708edaf0-471d-4dcd-9670-b6975a5807d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.545652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:00.557534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.558179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.558458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.558732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.569093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.569093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.569284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-708edaf0-471d-4dcd-9670-b6975a5807d6 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:00.569284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-708edaf0-471d-4dcd-9670-b6975a5807d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:00.570290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:00.571214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:00.577162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 492/993] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:00.580213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.626209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:00.633496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 502/994] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:00.634729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307571ab-d8db-4979-86c5-cb19e65dfb24 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.635016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307571ab-d8db-4979-86c5-cb19e65dfb24 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.635290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307571ab-d8db-4979-86c5-cb19e65dfb24 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.636720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 503/995] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:00.642278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef68c0-9f8b-4644-a010-61059c7a44c3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.642566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef68c0-9f8b-4644-a010-61059c7a44c3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.642843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef68c0-9f8b-4644-a010-61059c7a44c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.662055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-aeef68c0-9f8b-4644-a010-61059c7a44c3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:00.663247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-aeef68c0-9f8b-4644-a010-61059c7a44c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:00.688185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23787c-2219-4654-b76f-6f002a29b5e7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.721762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.722373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.722651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.722924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.723594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] RBAC: Adding query filter params (name=image) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:00.723895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.725299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.729939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 493/996] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:00.730695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.736007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 504/997] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:00.736858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef68c0-9f8b-4644-a010-61059c7a44c3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.784218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:00.832917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960be53-7871-4b01-8a59-d51f4a87a1df None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.838612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef68c0-9f8b-4644-a010-61059c7a44c3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.864225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:00.868019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.868690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.868975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.869874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.871235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.873110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.874004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.874340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.874619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.875301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:00.875626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.877006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:00.880108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:00.881180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:00.882167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 494/998] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:00.883436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 505/999] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:00.885299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.887409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.004653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad5b72-036d-468d-ae36-b32327c137d9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.008053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d267-dcc1-433b-bc46-a32c359fd121 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.024275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:01.047434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.048066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.048350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.048648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.049342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:01.049642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.051006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.060434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.060434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.060434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.060434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.060854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 506/1000] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:01.060939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.060939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.062072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:01.068519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:01.069534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:01.070383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 495/1001] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:01.076030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.120921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:01.167724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64c906-03ab-4f5e-bf64-98655475d4e5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.278882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f4804-1870-4ba4-9759-59bb27681c21 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.283312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.284130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.284614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.285040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.286287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:01.286594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.287972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.312338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.312973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.313250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.313517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.314183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:01.314482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.315838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.334273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 507/1002] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:01.334787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 496/1003] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:01.337243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.337355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.387170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:01.446939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7952df-46d9-4be2-8b09-a751e4bffd04 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.451566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661b8d9-5240-45b7-9662-b2f2aed5daae None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.469492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:01.487320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.488188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.488502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.488785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.489475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:01.489782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.491149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.493106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.493722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.494026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.494299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.495053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] RBAC: Adding query filter params (interface=public, service_id=978a9598eaca4ad9938bbb354198b98e, region_id=RegionOne) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:01.495359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.496546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 508/1004] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:01.496740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.501961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.506514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 497/1005] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:56:01 2022] GET /identity/v3/endpoints?service_id=978a9598eaca4ad9938bbb354198b98e&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:01.507361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.609428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad20ac8-6343-4ef1-b90d-75574df00f8c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.614486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c44d0-7633-4323-830e-be5ba7bdee8a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.622994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:01.644249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d199d-10ee-4d38-bd51-08f60176c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.644815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d199d-10ee-4d38-bd51-08f60176c98e None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.645094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d199d-10ee-4d38-bd51-08f60176c98e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.645365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d199d-10ee-4d38-bd51-08f60176c98e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.646076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d199d-10ee-4d38-bd51-08f60176c98e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.647429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d199d-10ee-4d38-bd51-08f60176c98e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.651050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c0964-12e5-4d54-b7d9-c2af17d6cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.651671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c0964-12e5-4d54-b7d9-c2af17d6cffd None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.651955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c0964-12e5-4d54-b7d9-c2af17d6cffd None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.652359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c0964-12e5-4d54-b7d9-c2af17d6cffd None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.654544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c0964-12e5-4d54-b7d9-c2af17d6cffd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.656736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c0964-12e5-4d54-b7d9-c2af17d6cffd None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.663286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 498/1006] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:01 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:56:01.714988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 509/1007] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:56:02.884682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186769b9-4183-4859-9f6a-569262ecb0be None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.885202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186769b9-4183-4859-9f6a-569262ecb0be None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.885331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186769b9-4183-4859-9f6a-569262ecb0be None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.886603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 499/1008] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:02.892149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801b5ae-71c7-4fbd-a6e2-3634cdc7d8ff None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.892552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801b5ae-71c7-4fbd-a6e2-3634cdc7d8ff None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.892752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801b5ae-71c7-4fbd-a6e2-3634cdc7d8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.911017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f801b5ae-71c7-4fbd-a6e2-3634cdc7d8ff None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:02.912203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f801b5ae-71c7-4fbd-a6e2-3634cdc7d8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:02.949275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:02.985501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 510/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:03.007626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186769b9-4183-4859-9f6a-569262ecb0be None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.123303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186769b9-4183-4859-9f6a-569262ecb0be None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:03.157227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c848e1-78d0-4579-9de1-6afafd1d9f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:03.158157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c848e1-78d0-4579-9de1-6afafd1d9f31 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.158671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c848e1-78d0-4579-9de1-6afafd1d9f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.159148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c848e1-78d0-4579-9de1-6afafd1d9f31 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.160326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c848e1-78d0-4579-9de1-6afafd1d9f31 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:03.161875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c848e1-78d0-4579-9de1-6afafd1d9f31 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:03.276415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 500/1010] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:03 2022] POST /identity/v3/services => generated 242 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:03.733452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af6eb3-44cb-4a58-b3fe-f626b34c37bc None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.733858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af6eb3-44cb-4a58-b3fe-f626b34c37bc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.734045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af6eb3-44cb-4a58-b3fe-f626b34c37bc None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.735413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 511/1011] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:03.740809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc46e-2eec-490a-adb0-07d316e9cf89 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.741242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc46e-2eec-490a-adb0-07d316e9cf89 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.741557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc46e-2eec-490a-adb0-07d316e9cf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.751782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d143c-bf05-4d59-98b2-149ed2dbbbc1 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.752065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d143c-bf05-4d59-98b2-149ed2dbbbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.752338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d143c-bf05-4d59-98b2-149ed2dbbbc1 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.753718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 512/1012] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:03.759243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e20e-53e1-45e6-a0e1-02ecb1eef9c2 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.759527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e20e-53e1-45e6-a0e1-02ecb1eef9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.759804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60e20e-53e1-45e6-a0e1-02ecb1eef9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.764364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-560cc46e-2eec-490a-adb0-07d316e9cf89 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:03.765576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-560cc46e-2eec-490a-adb0-07d316e9cf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:03.781012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ef60e20e-53e1-45e6-a0e1-02ecb1eef9c2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:03.782205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ef60e20e-53e1-45e6-a0e1-02ecb1eef9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:03.791478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:03.846292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 501/1013] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:03 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:03.860611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 513/1014] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:03 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:03.868991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cc46e-2eec-490a-adb0-07d316e9cf89 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.877751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e20e-53e1-45e6-a0e1-02ecb1eef9c2 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.979318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:03.982218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cc46e-2eec-490a-adb0-07d316e9cf89 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:03.985140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60e20e-53e1-45e6-a0e1-02ecb1eef9c2 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.020719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.020719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.020719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.020719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.021335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.022631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.025382 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.026028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.026310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.026584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.027860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.029337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.035889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 514/1015] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:04.036021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:04.037050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:04.037879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 502/1016] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:04.041208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.041319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.068507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.156846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77974c23-26d9-41db-a775-8ff3d0ec854a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.157603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6193f-3e0f-4948-8b15-fcd37cd36a65 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.194286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.194912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.195194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.195468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.196224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:04.196589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.197918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.199018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.199967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.200353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.200658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.201452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.203097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.203200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 503/1017] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:04.208624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.276712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 515/1018] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:04 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:04.309834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.327521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b39bde-5335-48d0-aa2f-da29550b2961 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.376000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cf167-350b-4db5-ba84-c1b99ef06774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.376675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cf167-350b-4db5-ba84-c1b99ef06774 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.376958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cf167-350b-4db5-ba84-c1b99ef06774 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.377233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cf167-350b-4db5-ba84-c1b99ef06774 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.377928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cf167-350b-4db5-ba84-c1b99ef06774 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:04.378234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cf167-350b-4db5-ba84-c1b99ef06774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.379642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cf167-350b-4db5-ba84-c1b99ef06774 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.390203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 504/1019] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:04.391630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.415371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.502428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d198-7c26-4014-a405-8b43f251ffe4 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.537830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dd215-77a2-489c-8462-0adbdcae9031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.538365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd215-77a2-489c-8462-0adbdcae9031 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.538650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd215-77a2-489c-8462-0adbdcae9031 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.538919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd215-77a2-489c-8462-0adbdcae9031 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.539632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd215-77a2-489c-8462-0adbdcae9031 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.541028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd215-77a2-489c-8462-0adbdcae9031 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.623986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 516/1020] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:05.327338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36505ff-3ae9-426e-9f7b-5d9a43511980 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.327885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36505ff-3ae9-426e-9f7b-5d9a43511980 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.327885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36505ff-3ae9-426e-9f7b-5d9a43511980 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.331887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 505/1021] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:05.334812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1692713-4507-449f-8886-142c128b23e8 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.335213 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1692713-4507-449f-8886-142c128b23e8 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.335399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1692713-4507-449f-8886-142c128b23e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.353758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d1692713-4507-449f-8886-142c128b23e8 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:05.354960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d1692713-4507-449f-8886-142c128b23e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:05.425616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.431242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 517/1022] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:05.454701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36505ff-3ae9-426e-9f7b-5d9a43511980 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.560073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36505ff-3ae9-426e-9f7b-5d9a43511980 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.594323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.594906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.595189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.595467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] PATH_INFO: `/v3/services/image` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.596093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.597478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:05.606987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:05.607914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:05.608791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:05.610427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 506/1023] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:05.614248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1692713-4507-449f-8886-142c128b23e8 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.714458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1692713-4507-449f-8886-142c128b23e8 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.749724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.750341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.750629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.750901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.751581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.751883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.753271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.763060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 518/1024] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:05.764287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.781403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.870867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa1f12-431f-4b00-b2ac-1cbba5e86695 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.904448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.905066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.905341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.905609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.906278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] RBAC: Adding query filter params (type=image) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.906576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.907921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.917826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 507/1025] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:05.919312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.983008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:06.020462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62841c17-9020-4c53-8090-43864bae07d2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:06.054236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c48fc-5b86-4a97-ad53-dab30aceeeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:06.054851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c48fc-5b86-4a97-ad53-dab30aceeeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.055130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c48fc-5b86-4a97-ad53-dab30aceeeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.055401 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c48fc-5b86-4a97-ad53-dab30aceeeb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.056158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c48fc-5b86-4a97-ad53-dab30aceeeb4 None admin] RBAC: Adding query filter params (interface=public, service_id=6b29488acd4843c78bc0cd4db9d55c3d, region_id=RegionOne) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:06.056494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c48fc-5b86-4a97-ad53-dab30aceeeb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:06.057848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c48fc-5b86-4a97-ad53-dab30aceeeb4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:06.063495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 519/1026] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/endpoints?service_id=6b29488acd4843c78bc0cd4db9d55c3d&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:06.068862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:06.148536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:06.167144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cad726-a815-4529-88db-5e8e351bf968 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:06.201557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee05c6e-2349-46f4-b6e7-4c4784e69349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:06.202187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee05c6e-2349-46f4-b6e7-4c4784e69349 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.202468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee05c6e-2349-46f4-b6e7-4c4784e69349 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.202740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee05c6e-2349-46f4-b6e7-4c4784e69349 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.203394 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee05c6e-2349-46f4-b6e7-4c4784e69349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:06.204781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee05c6e-2349-46f4-b6e7-4c4784e69349 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:06.211660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 508/1027] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:06 2022] GET /identity/v3/services => generated 1957 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:56:06.354472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33805b09-81bb-4418-b3cb-fc5b52080aac None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.355006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33805b09-81bb-4418-b3cb-fc5b52080aac None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.355006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33805b09-81bb-4418-b3cb-fc5b52080aac None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.357728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 520/1028] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:06.362350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bb75-4fa6-44c7-8148-83e05ff09cd2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.362753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bb75-4fa6-44c7-8148-83e05ff09cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.362901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556bb75-4fa6-44c7-8148-83e05ff09cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.381930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-0556bb75-4fa6-44c7-8148-83e05ff09cd2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:06.383396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0556bb75-4fa6-44c7-8148-83e05ff09cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:06.453418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 509/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:06.477882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33805b09-81bb-4418-b3cb-fc5b52080aac None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:06.582414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33805b09-81bb-4418-b3cb-fc5b52080aac None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:06.609703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:06.617924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e15b33-79fe-411a-ba2b-8e1216203ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:06.618565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e15b33-79fe-411a-ba2b-8e1216203ccb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.618895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e15b33-79fe-411a-ba2b-8e1216203ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.619175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e15b33-79fe-411a-ba2b-8e1216203ccb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.620325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e15b33-79fe-411a-ba2b-8e1216203ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:06.624989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e15b33-79fe-411a-ba2b-8e1216203ccb None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:06.624989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 521/1030] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:06.629532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556bb75-4fa6-44c7-8148-83e05ff09cd2 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:06.701236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:06.717437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f54762-e82f-45ea-907b-03d56ab4eeaa None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.717701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f54762-e82f-45ea-907b-03d56ab4eeaa None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.717916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f54762-e82f-45ea-907b-03d56ab4eeaa None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.719272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 522/1031] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:06.724790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81585bd-f937-46f5-8b64-74e2b862bd88 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.725141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81585bd-f937-46f5-8b64-74e2b862bd88 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.725390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81585bd-f937-46f5-8b64-74e2b862bd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.734881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556bb75-4fa6-44c7-8148-83e05ff09cd2 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:06.746217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a81585bd-f937-46f5-8b64-74e2b862bd88 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:06.747424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a81585bd-f937-46f5-8b64-74e2b862bd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:06.773678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:06.774570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.774852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.775126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.775815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:06.777816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:06.816852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 523/1032] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:06.840770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81585bd-f937-46f5-8b64-74e2b862bd88 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:06.858179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 510/1033] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:56:06 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:06.953037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81585bd-f937-46f5-8b64-74e2b862bd88 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:06.981532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:06.990823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:06.991485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.000558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.000558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.003591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:07.009481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:07.010625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:07.012014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 524/1034] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:07.017423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.124584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb90ad7-0dbe-4323-b895-96ed1979ff5c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.150699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.159816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.160459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.160743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.161011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.161728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:07.162026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.163380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.172153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 511/1035] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:07.177656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.280528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd8a41-4d74-40b2-a4e4-978dc0eccb38 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.315152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.315794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.316075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.316349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.327913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.329428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:07.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:07.350292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:07.351211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:07.351211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:07.351211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:07.351211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:07.351211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 525/1036] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:07.357147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.460786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cde588-73ca-4ad7-8aec-a308f48384d7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.494793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.495412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.495693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.496543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.497240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:07.497544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.498934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.510152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 512/1037] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:07.515678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.558231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.618335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b70af-3f1e-4f84-90d7-61af60cbb010 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.652347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.653015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.653298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.653571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.661457 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.662907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:07.674788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:07.675741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:07.676745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:07.680464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 526/1038] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:07.682038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.733628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.785227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7785bd8-d555-4fb5-ad0a-993237740eb9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.822810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.822810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.823235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.823409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.824246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:07.824639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.826050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.844085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 513/1039] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:07.849472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.946532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.950947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab186f-f9a2-4a6f-bc04-acd9bfb8cd9d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.985823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab4615-ed8d-47bb-8cff-1ec43fc0cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.986436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab4615-ed8d-47bb-8cff-1ec43fc0cccb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.986711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab4615-ed8d-47bb-8cff-1ec43fc0cccb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.986981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab4615-ed8d-47bb-8cff-1ec43fc0cccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.057700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab4615-ed8d-47bb-8cff-1ec43fc0cccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.059653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab4615-ed8d-47bb-8cff-1ec43fc0cccb None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.159908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.169966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 527/1040] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:08.171537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.242039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed026a-c110-47e9-9cc8-2ffd9195031f None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.242397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed026a-c110-47e9-9cc8-2ffd9195031f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.242594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed026a-c110-47e9-9cc8-2ffd9195031f None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.244132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 528/1041] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:08.249243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d2eab-95f5-42ad-a01b-cd8d27cd264c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.249536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d2eab-95f5-42ad-a01b-cd8d27cd264c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.249814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d2eab-95f5-42ad-a01b-cd8d27cd264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.267181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b29d2eab-95f5-42ad-a01b-cd8d27cd264c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:08.268369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b29d2eab-95f5-42ad-a01b-cd8d27cd264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:08.275582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d101ee5-8bf4-4912-b0f0-0d2a617e3fcb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.283433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.314615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.315363 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.315646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.316032 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.326743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.328180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.344426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 529/1042] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:08.344588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:08.345430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:08.346407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:08.346407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:08.346407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:08.346407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:08.349464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 514/1043] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:08.350830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d2eab-95f5-42ad-a01b-cd8d27cd264c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.364836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.453810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d2eab-95f5-42ad-a01b-cd8d27cd264c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.475831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81c98-a52b-41d5-902e-c0616d49a186 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.484851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.496537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.499304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.499304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.499304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.499304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.499304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.499976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.511522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 530/1044] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:08.517154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.524750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.525410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.525677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.525951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] PATH_INFO: `/v3/services/image` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.527032 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.528767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:08.538950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:08.540068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:08.541081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:08.543974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 515/1045] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:08.549559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.574990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.624107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a1a2-f59b-4c15-a848-ff764d6cc070 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.657169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36af1e-5df2-4b4d-a6e0-a5d40e881676 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.662351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.662994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.663278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.663554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.671288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.672757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:08.684871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:08.685777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:08.687082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 531/1046] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:08.692271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.699635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.700255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.700571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.701176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.701534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] RBAC: Adding query filter params (name=image) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.701836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.703410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.712915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 516/1047] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:08.713877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.746661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.802490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1439f0-c268-4926-93f1-7eff84d318ff None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.819286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.837159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d152a2d-18a2-4807-9b68-23836a50a59b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.837449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.838060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.838450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.838731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.840005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.840313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.841786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.854230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 532/1048] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:08.860705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.873499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.874374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.874884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.875332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.876207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.876713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.878073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.887871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 517/1049] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:08.888574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.967411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6875559-8b5a-44d1-a39f-06be41af1e8e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.978249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.994127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54537ea2-c2be-434e-936d-2a0fe1700ed9 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.002079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b43c6c-566f-4e83-9bb1-9db26bd4301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.002702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43c6c-566f-4e83-9bb1-9db26bd4301b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.002980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43c6c-566f-4e83-9bb1-9db26bd4301b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.003253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43c6c-566f-4e83-9bb1-9db26bd4301b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.004044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43c6c-566f-4e83-9bb1-9db26bd4301b None admin] RBAC: Adding query filter params (role.id=0ded57e2af4f44de838c73c41429336f, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=784880e276bf4d64ae785609a463403a) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:09.004414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43c6c-566f-4e83-9bb1-9db26bd4301b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.006462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43c6c-566f-4e83-9bb1-9db26bd4301b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.012733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 533/1050] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/role_assignments?user.id=784880e276bf4d64ae785609a463403a&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=0ded57e2af4f44de838c73c41429336f => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:09.015119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6a111-c3e0-42c1-a911-443c8d7366a0 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.015410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6a111-c3e0-42c1-a911-443c8d7366a0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.015687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6a111-c3e0-42c1-a911-443c8d7366a0 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.017051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 534/1051] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:09.022924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43fb13-e039-4c1c-904d-99994c777796 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.023258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43fb13-e039-4c1c-904d-99994c777796 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.023485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43fb13-e039-4c1c-904d-99994c777796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.029566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.030215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.030542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.030826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.031545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.032987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.044288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-5f43fb13-e039-4c1c-904d-99994c777796 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:09.045564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-5f43fb13-e039-4c1c-904d-99994c777796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:09.111461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 518/1052] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:09.134334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 535/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:09.160298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.171409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.275106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31fb5a-95b6-4a96-bf9a-ea42fafbfbea None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.310738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.311379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.311659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.311930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.322884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:09.331754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:09.332733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:09.333569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 519/1054] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:09.336160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43fb13-e039-4c1c-904d-99994c777796 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.439524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.443896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43fb13-e039-4c1c-904d-99994c777796 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.478974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.479586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.479895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.480164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.480905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:09.481204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.482555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.490922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 536/1055] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:09.497355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.562840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.601217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d065-7e84-4c29-9048-82f8707c7d3e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.638775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.639789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.640298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.640815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] PATH_INFO: `/v3/users/d531a8e813ac4d07aca61ff1791c674d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.655633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] RBAC: Authorizing `identity:get_user(user_id=d531a8e813ac4d07aca61ff1791c674d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.657392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.659532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 520/1056] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/users/d531a8e813ac4d07aca61ff1791c674d => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:09.665334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.771368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281b0c6-e3d8-4ea5-8654-01ebf8d295dc None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.804849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.805462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.805740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.806008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.833412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.872517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.874167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.974667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 537/1057] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:09.975852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.077795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b28610-a941-4189-aafb-2e054e0c632e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.101962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:10.114230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933b78b-fd34-410d-9123-57e1aedb40b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.115256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933b78b-fd34-410d-9123-57e1aedb40b7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.115706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933b78b-fd34-410d-9123-57e1aedb40b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.116148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933b78b-fd34-410d-9123-57e1aedb40b7 None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.120252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933b78b-fd34-410d-9123-57e1aedb40b7 None admin] RBAC: Authorizing `identity:get_project(project_id=262b85d39a3842e8aeb0ba1424ba632a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.120252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933b78b-fd34-410d-9123-57e1aedb40b7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.123323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 521/1058] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:10.127555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.189404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:10.231340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c4cfe-345c-451c-901e-e0b299eda039 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.265333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854eb0f-3b47-4381-933a-aee1e9ce50c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.265976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854eb0f-3b47-4381-933a-aee1e9ce50c3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.266258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854eb0f-3b47-4381-933a-aee1e9ce50c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.266534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854eb0f-3b47-4381-933a-aee1e9ce50c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.267350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854eb0f-3b47-4381-933a-aee1e9ce50c3 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=262b85d39a3842e8aeb0ba1424ba632a, user.id=d531a8e813ac4d07aca61ff1791c674d) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:10.267664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854eb0f-3b47-4381-933a-aee1e9ce50c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.269174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854eb0f-3b47-4381-933a-aee1e9ce50c3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.280407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 538/1059] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:10 2022] GET /identity/v3/role_assignments?user.id=d531a8e813ac4d07aca61ff1791c674d&scope.project.id=262b85d39a3842e8aeb0ba1424ba632a&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:11.013021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc883d4-4479-4e97-832b-d4b16a9ac84d None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.013557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc883d4-4479-4e97-832b-d4b16a9ac84d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.013557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc883d4-4479-4e97-832b-d4b16a9ac84d None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.014895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 522/1060] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:11.020215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb339d32-4e9e-4a1a-af64-4519b0cd8bbd None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.020683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb339d32-4e9e-4a1a-af64-4519b0cd8bbd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.020866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb339d32-4e9e-4a1a-af64-4519b0cd8bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.039453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-cb339d32-4e9e-4a1a-af64-4519b0cd8bbd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:11.040667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cb339d32-4e9e-4a1a-af64-4519b0cd8bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:11.107967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 539/1061] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:11.130618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc883d4-4479-4e97-832b-d4b16a9ac84d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.175589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33001460-626d-4594-8541-3d570c79251c None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.175589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33001460-626d-4594-8541-3d570c79251c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.175589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33001460-626d-4594-8541-3d570c79251c None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.175589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 540/1062] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:11.180779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0f7c-ebf0-45a7-aff2-fb1c45547f1b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.181278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0f7c-ebf0-45a7-aff2-fb1c45547f1b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.181748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a0f7c-ebf0-45a7-aff2-fb1c45547f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.201383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-704a0f7c-ebf0-45a7-aff2-fb1c45547f1b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:11.202839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-704a0f7c-ebf0-45a7-aff2-fb1c45547f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:11.235146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc883d4-4479-4e97-832b-d4b16a9ac84d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.255830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.269976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.270664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.270942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.271210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.276410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 541/1063] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:11.281817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:11.287173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:11.288084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:11.288937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:11.292721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 523/1064] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:11.294713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a0f7c-ebf0-45a7-aff2-fb1c45547f1b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.296880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.359928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.404370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a0f7c-ebf0-45a7-aff2-fb1c45547f1b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.411739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329491-d2e3-49c7-9604-2ebde08bc69c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.443875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.445065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.445518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.445966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.446854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:11.447358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.448193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.448851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.449131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.449403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.449747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.460236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 542/1065] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:11.465561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.516949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.526570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.526570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.572971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7f429-3986-4c46-9ca2-95983352d8e6 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.578162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.612231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.612901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.613184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.613456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.625011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.626448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.635208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 524/1066] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:56:11.641307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:11.645878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:11.646746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:11.647640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:11.647640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:11.647640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:11.647640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:11.647640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 543/1067] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:11.653187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.748664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266de87-5687-40fa-ba64-8f38c0453832 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.757215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.761362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d145ab6-456c-4902-a5f6-61588eb70289 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.785905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.786482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.786805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.787114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.797561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.798176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.798454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.798568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.798724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.799411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:11.799712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.800012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.801082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.813871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 544/1068] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:11.815157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:11.816007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:11.821791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:11.821791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:11.821791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:11.821791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:11.821791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 525/1069] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:11.822076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.822148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.826891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.930003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0c3f0-47ac-4552-b2de-070f4e452bef None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.936632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46695ec-1587-4d67-9f3b-b5f32f15d889 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.967846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.968519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.968802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.969075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.971176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.971808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.972148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.972458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.973157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:11.973463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.974827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.976624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.978051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.987469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 526/1070] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:11.990016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:11.990927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:11.996900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 545/1071] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:11.999071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.068482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.109950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ecb09-f441-4fc8-983e-2c405d4fc102 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.147980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeb70d-e1ff-4318-b394-a65e5d0e45ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.148628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb70d-e1ff-4318-b394-a65e5d0e45ed None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.148913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb70d-e1ff-4318-b394-a65e5d0e45ed None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.149173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb70d-e1ff-4318-b394-a65e5d0e45ed None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.149973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeb70d-e1ff-4318-b394-a65e5d0e45ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:12.150273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeb70d-e1ff-4318-b394-a65e5d0e45ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.152284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaeb70d-e1ff-4318-b394-a65e5d0e45ed None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.170548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 527/1072] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:12.176064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.196710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.287488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffca1a4-43d7-4b74-bc53-9907752f057a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.291565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c1267-067a-4d6a-b111-47891dd2e003 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.291850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c1267-067a-4d6a-b111-47891dd2e003 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.292128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c1267-067a-4d6a-b111-47891dd2e003 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.293487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 528/1073] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:12.298471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877d4da-b58b-4646-90db-88c57bc03aac None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.298757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877d4da-b58b-4646-90db-88c57bc03aac None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.299028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877d4da-b58b-4646-90db-88c57bc03aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.317232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8877d4da-b58b-4646-90db-88c57bc03aac None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:12.318406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8877d4da-b58b-4646-90db-88c57bc03aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:12.321499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.322072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.322350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.322622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.334646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.336548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:12.350975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:12.351893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:12.352819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:12.352819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:12.352819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:12.352819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:12.352819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 546/1074] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:12.357723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.390031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 529/1075] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:12.412048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877d4da-b58b-4646-90db-88c57bc03aac None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.427054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.458001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b6f78-f61f-4174-b0f0-612b91a146fb None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.464412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.491984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.492643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.492931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.493210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.493907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:12.494215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.496093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.507600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 547/1076] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:12.511802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877d4da-b58b-4646-90db-88c57bc03aac None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.513822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.545143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.545788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.546072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.546348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.556305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:12.561811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:12.562920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:12.564112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 530/1077] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:12.568789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.612822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c735b-58b7-4591-b226-d0f32593540a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.641589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.647074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.647798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.648072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.648342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.656947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.658637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.668039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850c293-13b0-4897-a77a-5398f12de765 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:12.670531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:12.671577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:12.672669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 548/1078] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:12.677504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.683186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.703575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.704192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.704501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.704780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.705492 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:12.705795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.707159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.716161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 531/1079] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:12.721893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.775120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca45b5-59db-4129-ac61-d1c76878e240 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.808798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.809415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.809701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.809976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.810732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:12.811043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.812494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.818637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910cd36-927f-4b79-81dd-5978032aa2d7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.824098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 549/1080] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:12.830314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.852302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.852973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.853257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.853533 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] PATH_INFO: `/v3/users/d531a8e813ac4d07aca61ff1791c674d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.855228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] RBAC: Authorizing `identity:get_user(user_id=d531a8e813ac4d07aca61ff1791c674d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.856677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.858408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 532/1081] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/users/d531a8e813ac4d07aca61ff1791c674d => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:12.863787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.882507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.905053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.931950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecea9c-f312-4c72-9adc-e36cd47f2dc5 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.961975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333fbab-395a-49a8-9d70-48d3e6ac776a None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.966892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b34a3-fc66-4abd-8775-ec5da95062c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.967545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b34a3-fc66-4abd-8775-ec5da95062c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.967819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b34a3-fc66-4abd-8775-ec5da95062c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.968090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b34a3-fc66-4abd-8775-ec5da95062c8 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/784880e276bf4d64ae785609a463403a/roles/0ded57e2af4f44de838c73c41429336f` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.970293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b34a3-fc66-4abd-8775-ec5da95062c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=784880e276bf4d64ae785609a463403a, role_id=0ded57e2af4f44de838c73c41429336f)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.971702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b34a3-fc66-4abd-8775-ec5da95062c8 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.996034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.996690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.996966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.997236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.998443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] RBAC: Authorizing `identity:get_project(project_id=262b85d39a3842e8aeb0ba1424ba632a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.999836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.001510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 533/1082] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:13.007418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.073632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 550/1083] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:12 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/784880e276bf4d64ae785609a463403a/roles/0ded57e2af4f44de838c73c41429336f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:56:13.106044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab63cf-23cc-4022-876c-f61829064a51 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.117832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:13.143094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c877dc7-2463-4322-a7e0-cfbd38c5bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.143808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c877dc7-2463-4322-a7e0-cfbd38c5bd52 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.144082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c877dc7-2463-4322-a7e0-cfbd38c5bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.144350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c877dc7-2463-4322-a7e0-cfbd38c5bd52 None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a/users/d531a8e813ac4d07aca61ff1791c674d/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.146359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c877dc7-2463-4322-a7e0-cfbd38c5bd52 None admin] RBAC: Authorizing `identity:create_grant(project_id=262b85d39a3842e8aeb0ba1424ba632a, user_id=d531a8e813ac4d07aca61ff1791c674d, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.147873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c877dc7-2463-4322-a7e0-cfbd38c5bd52 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.219915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 534/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:13 2022] PUT /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a/users/d531a8e813ac4d07aca61ff1791c674d/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:56:15.065333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3640-cf5d-42b6-b2d4-942308667c34 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.065863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3640-cf5d-42b6-b2d4-942308667c34 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.065863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3640-cf5d-42b6-b2d4-942308667c34 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.067310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 551/1085] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:15.073050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620993-ed3b-440a-8c5b-9ca3f7484d79 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.073454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620993-ed3b-440a-8c5b-9ca3f7484d79 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.073599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620993-ed3b-440a-8c5b-9ca3f7484d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.091837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-75620993-ed3b-440a-8c5b-9ca3f7484d79 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:15.093038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-75620993-ed3b-440a-8c5b-9ca3f7484d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:15.159100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 535/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:15.182188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd3640-cf5d-42b6-b2d4-942308667c34 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.191910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bac196-7301-4290-961b-23b1f83a5511 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.192197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bac196-7301-4290-961b-23b1f83a5511 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.192652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bac196-7301-4290-961b-23b1f83a5511 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.193990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 536/1087] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:15.199399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf9715-16f7-499e-9541-98d8c1a1c4ee None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.199684 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf9715-16f7-499e-9541-98d8c1a1c4ee None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.199962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf9715-16f7-499e-9541-98d8c1a1c4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.217765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e7bf9715-16f7-499e-9541-98d8c1a1c4ee None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:15.218951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e7bf9715-16f7-499e-9541-98d8c1a1c4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:15.285490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd3640-cf5d-42b6-b2d4-942308667c34 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.288386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 537/1088] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:15.300267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:15.311983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf9715-16f7-499e-9541-98d8c1a1c4ee None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.324592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.325163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.325444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.325718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.336544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:15.341509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:15.342482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:15.345191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 552/1089] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:15.346578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:15.348515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.415802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf9715-16f7-499e-9541-98d8c1a1c4ee None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.446319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72465e-7124-48c2-a374-6f976008a70c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.450160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.450796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.451076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.451349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.461281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:15.466267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:15.467313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:15.468131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:15.472429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 538/1090] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:15.473265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.479896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.480543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.480826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.481098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.481814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:15.482117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.483486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.496356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 553/1091] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:15.497341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.527931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:15.562301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:15.571792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f798cd-4d59-4890-81c0-411f90b05051 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.596405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb7bfd-ea27-41ce-a175-53a1eaa60ee0 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.606470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.607080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.607359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.607632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.608340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:15.608673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.610034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.618310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 539/1092] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:15.624647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.628891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.629518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.629794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.630063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.643370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.644850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:15.659887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:15.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:15.661935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:15.661935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:15.661935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:15.661935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:15.661935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 554/1093] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:15.667310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.724155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f4711-124a-4b13-97e5-fc5c300b14ae None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.751806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:15.757550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.758190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.758475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.758764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] PATH_INFO: `/v3/users/d531a8e813ac4d07aca61ff1791c674d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.760518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=d531a8e813ac4d07aca61ff1791c674d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.761944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.763541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 540/1094] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/users/d531a8e813ac4d07aca61ff1791c674d => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:15.768094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949e6cb-2a41-479c-83fc-0191034efa71 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.769205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.791036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:15.803172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.803782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.804063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.804334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.805051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:15.805355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.806734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.818177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 555/1095] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:15.824143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.870596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3c72-21f1-4836-a189-6652e9975ae3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.903836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.904486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.904770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.905047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.922895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c305d0-b811-4d7f-9a5d-22afb7bf28ff None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.956553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.957175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.957458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.957733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.965113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.966549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.971258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.972703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:15.978403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:15.979339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:15.980266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 556/1096] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:15.985230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.988636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:15.992923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.071909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 541/1097] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:56:16.078626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.083698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f7248-10b7-4e58-aa26-053388fc88cf None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.116702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030913-e41a-46e7-9d21-268745275402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.117306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030913-e41a-46e7-9d21-268745275402 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.117582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030913-e41a-46e7-9d21-268745275402 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.117858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04030913-e41a-46e7-9d21-268745275402 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.118645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04030913-e41a-46e7-9d21-268745275402 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:16.119431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04030913-e41a-46e7-9d21-268745275402 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.120922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04030913-e41a-46e7-9d21-268745275402 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.138439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 557/1098] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:16.143812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030913-e41a-46e7-9d21-268745275402 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.176607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061855b-3c3f-4d07-941f-f93191d01b30 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.197710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.203617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.210993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.211631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.211908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.212182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.213462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] RBAC: Authorizing `identity:get_project(project_id=262b85d39a3842e8aeb0ba1424ba632a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.214876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.216707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 542/1099] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:16.222472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.240843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04030913-e41a-46e7-9d21-268745275402 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.276157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.276813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.277097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.277375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.321125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abbcbc-0f86-4d76-ae2c-177be109daa8 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.343862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.345347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.358795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.359411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.359687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.359951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.360756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=262b85d39a3842e8aeb0ba1424ba632a, user.id=d531a8e813ac4d07aca61ff1791c674d) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:16.361061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.362463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.368350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 543/1100] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/role_assignments?user.id=d531a8e813ac4d07aca61ff1791c674d&scope.project.id=262b85d39a3842e8aeb0ba1424ba632a&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:16.407849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.449622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 558/1101] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:56:16.455603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.510903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.559602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be48067-ee6a-410c-9a3f-44719cb542c3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.592678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.593318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.593599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.593873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.604794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.606226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:16.621155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:16.622037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:16.622937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:16.622937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:16.622937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:16.622937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:16.622937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 544/1102] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:16.627974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.726700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96778a7d-494d-4d04-a469-8eb9dd7c5f8f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.760168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.760810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.761083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.761351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.762038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:16.762339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.763694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.774387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 559/1103] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:16.780004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.876628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1dab0-dabd-408e-a84e-22489dd1c3ab None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.878966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.909434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.910067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.910386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.910629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.917693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.919632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:16.931310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:16.932161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:16.933107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 545/1104] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:16.938221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.963253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.036792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70930-cdb5-4f40-95a5-699bbd8d266e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.068855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c06caf-5552-4c0a-8baf-728fa9da900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.069463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c06caf-5552-4c0a-8baf-728fa9da900f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.069740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c06caf-5552-4c0a-8baf-728fa9da900f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.070009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c06caf-5552-4c0a-8baf-728fa9da900f None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.070750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c06caf-5552-4c0a-8baf-728fa9da900f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.071053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c06caf-5552-4c0a-8baf-728fa9da900f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.072547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c06caf-5552-4c0a-8baf-728fa9da900f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.084581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 560/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:17.091026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.232991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7645-1e72-4fe4-884d-eb80da857321 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.265310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76dbe0-8622-461c-a1cf-ed4b3be0e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.265914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76dbe0-8622-461c-a1cf-ed4b3be0e0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.266190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76dbe0-8622-461c-a1cf-ed4b3be0e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.266456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76dbe0-8622-461c-a1cf-ed4b3be0e0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.267224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76dbe0-8622-461c-a1cf-ed4b3be0e0d1 None admin] RBAC: Adding query filter params (role.id=0ded57e2af4f44de838c73c41429336f, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=784880e276bf4d64ae785609a463403a) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.267525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76dbe0-8622-461c-a1cf-ed4b3be0e0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.268960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76dbe0-8622-461c-a1cf-ed4b3be0e0d1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.274599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 546/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/role_assignments?user.id=784880e276bf4d64ae785609a463403a&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=0ded57e2af4f44de838c73c41429336f => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:18.401691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f189aae-5fc2-4348-bea5-bd13cf6eb2c9 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:18.402209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f189aae-5fc2-4348-bea5-bd13cf6eb2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:18.402303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f189aae-5fc2-4348-bea5-bd13cf6eb2c9 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:18.403593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 561/1107] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:18.409013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fb7d-7263-42e3-a59f-94d4e1499c02 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:18.409425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fb7d-7263-42e3-a59f-94d4e1499c02 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:18.409546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3fb7d-7263-42e3-a59f-94d4e1499c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:18.427110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-9df3fb7d-7263-42e3-a59f-94d4e1499c02 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:18.428286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9df3fb7d-7263-42e3-a59f-94d4e1499c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:18.451062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:18.497878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 547/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:18.523523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f189aae-5fc2-4348-bea5-bd13cf6eb2c9 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:18.604949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:18.623818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f189aae-5fc2-4348-bea5-bd13cf6eb2c9 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:18.657532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750fd74-c9cf-476a-b0f1-3991c38eff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:18.658158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750fd74-c9cf-476a-b0f1-3991c38eff4d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:18.658439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750fd74-c9cf-476a-b0f1-3991c38eff4d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:18.658715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750fd74-c9cf-476a-b0f1-3991c38eff4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:18.659861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750fd74-c9cf-476a-b0f1-3991c38eff4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:18.661323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750fd74-c9cf-476a-b0f1-3991c38eff4d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:18.662897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 562/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:18.669255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3fb7d-7263-42e3-a59f-94d4e1499c02 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:18.763739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3fb7d-7263-42e3-a59f-94d4e1499c02 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:18.796654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c67e4b-553f-4e40-b970-ec5cfa98a7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:18.797262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c67e4b-553f-4e40-b970-ec5cfa98a7c5 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:18.797543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c67e4b-553f-4e40-b970-ec5cfa98a7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:18.797818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c67e4b-553f-4e40-b970-ec5cfa98a7c5 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:18.798491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c67e4b-553f-4e40-b970-ec5cfa98a7c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:18.799903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c67e4b-553f-4e40-b970-ec5cfa98a7c5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:18.884443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 548/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:56:18 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:20.915591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f8d20-7aee-4295-aae6-770cf3293fb9 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.916123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f8d20-7aee-4295-aae6-770cf3293fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.916123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f8d20-7aee-4295-aae6-770cf3293fb9 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.917803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 563/1111] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:20.923086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6515-4d9a-4e4d-90d8-d4935ed6dd14 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.923505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6515-4d9a-4e4d-90d8-d4935ed6dd14 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.923632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c6515-4d9a-4e4d-90d8-d4935ed6dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.941692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-216c6515-4d9a-4e4d-90d8-d4935ed6dd14 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:20.942858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-216c6515-4d9a-4e4d-90d8-d4935ed6dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:20.951781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:21.006658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 549/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:21.029360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f8d20-7aee-4295-aae6-770cf3293fb9 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:21.128028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f8d20-7aee-4295-aae6-770cf3293fb9 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:21.160389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:21.161033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:21.161315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:21.161588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] PATH_INFO: `/v3/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:21.163059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] RBAC: Authorizing `identity:get_role(role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:21.164438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:21.166395 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 564/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:21 2022] GET /identity/v3/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:21.172133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6515-4d9a-4e4d-90d8-d4935ed6dd14 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:21.267460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c6515-4d9a-4e4d-90d8-d4935ed6dd14 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:21.300581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:21.301201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:21.301477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:21.301747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] PATH_INFO: `/v3/users/0002c704d39644778da298532e4f9ba2` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:21.314838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] RBAC: Authorizing `identity:get_user(user_id=0002c704d39644778da298532e4f9ba2)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:21.316250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:21.317887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 550/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:21 2022] GET /identity/v3/users/0002c704d39644778da298532e4f9ba2 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:21.323581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:21.342655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:21.420987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774f538-f253-4268-8e9a-1fa732599da1 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:21.453038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:21.453634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:21.453911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:21.454182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:21.518941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:21.520367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:21.537925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:21.614679 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 565/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:21.620582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:21.623422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:21.716707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f697f9-a819-45b5-99f6-eac896b89342 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:21.752938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44d5a1-ede1-41ce-a6bd-38de90ef38f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:21.753565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44d5a1-ede1-41ce-a6bd-38de90ef38f8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:21.753841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44d5a1-ede1-41ce-a6bd-38de90ef38f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:21.754117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44d5a1-ede1-41ce-a6bd-38de90ef38f8 None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:21.755253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44d5a1-ede1-41ce-a6bd-38de90ef38f8 None admin] RBAC: Authorizing `identity:get_project(project_id=262b85d39a3842e8aeb0ba1424ba632a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:21.756692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44d5a1-ede1-41ce-a6bd-38de90ef38f8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:21.758852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 551/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:21 2022] GET /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:21.765138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:21.861246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfbe65-23a3-4e63-8db9-faa96e67e48c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:21.877783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:21.894214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea662b63-7a91-4439-97ee-a19cbc1c2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:21.894827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea662b63-7a91-4439-97ee-a19cbc1c2591 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:21.895114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea662b63-7a91-4439-97ee-a19cbc1c2591 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:21.895390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea662b63-7a91-4439-97ee-a19cbc1c2591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:21.896189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea662b63-7a91-4439-97ee-a19cbc1c2591 None admin] RBAC: Adding query filter params (role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=262b85d39a3842e8aeb0ba1424ba632a, user.id=0002c704d39644778da298532e4f9ba2) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:21.896541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea662b63-7a91-4439-97ee-a19cbc1c2591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:21.898104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea662b63-7a91-4439-97ee-a19cbc1c2591 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:21.904198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 566/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:21 2022] GET /identity/v3/role_assignments?user.id=0002c704d39644778da298532e4f9ba2&scope.project.id=262b85d39a3842e8aeb0ba1424ba632a&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:23.850785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb859025-053b-4eb2-8229-83beb7828ec2 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.851313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb859025-053b-4eb2-8229-83beb7828ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.851313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb859025-053b-4eb2-8229-83beb7828ec2 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.852715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 552/1118] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:23.857715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca993c-354f-4c67-8bd1-a2c4ece62245 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.858095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca993c-354f-4c67-8bd1-a2c4ece62245 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.858266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca993c-354f-4c67-8bd1-a2c4ece62245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.876417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3cca993c-354f-4c67-8bd1-a2c4ece62245 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:23.877603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3cca993c-354f-4c67-8bd1-a2c4ece62245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:23.944089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 567/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:23.965910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb859025-053b-4eb2-8229-83beb7828ec2 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.033965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:24.064443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb859025-053b-4eb2-8229-83beb7828ec2 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.097072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.097696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.097978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.098254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] PATH_INFO: `/v3/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.099688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] RBAC: Authorizing `identity:get_role(role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.101123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.102900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 553/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:24.108665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca993c-354f-4c67-8bd1-a2c4ece62245 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.203937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca993c-354f-4c67-8bd1-a2c4ece62245 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.216514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:24.236934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.237566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.237848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.238119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] PATH_INFO: `/v3/users/0002c704d39644778da298532e4f9ba2` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.239258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] RBAC: Authorizing `identity:get_user(user_id=0002c704d39644778da298532e4f9ba2)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.240833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.243031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 568/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:24 2022] GET /identity/v3/users/0002c704d39644778da298532e4f9ba2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:24.248827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.344499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a4d26-3a1a-4b4c-ba21-faabed950dc6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.377375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e3c26-732a-48a9-8284-17d8aebb1037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.377938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e3c26-732a-48a9-8284-17d8aebb1037 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.378218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e3c26-732a-48a9-8284-17d8aebb1037 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.378493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e3c26-732a-48a9-8284-17d8aebb1037 None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.379630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e3c26-732a-48a9-8284-17d8aebb1037 None admin] RBAC: Authorizing `identity:get_project(project_id=262b85d39a3842e8aeb0ba1424ba632a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.381078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e3c26-732a-48a9-8284-17d8aebb1037 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.382740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 554/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:24 2022] GET /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:24.388593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.486726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820df31-522e-4461-9a51-5f68b398b8ce None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.519103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2929d92-c86a-4c18-8d9f-2ba62bc402ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.519758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929d92-c86a-4c18-8d9f-2ba62bc402ec None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.520040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929d92-c86a-4c18-8d9f-2ba62bc402ec None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.520318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929d92-c86a-4c18-8d9f-2ba62bc402ec None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a/users/0002c704d39644778da298532e4f9ba2/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.522518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2929d92-c86a-4c18-8d9f-2ba62bc402ec None admin] RBAC: Authorizing `identity:create_grant(project_id=262b85d39a3842e8aeb0ba1424ba632a, user_id=0002c704d39644778da298532e4f9ba2, role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.524488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2929d92-c86a-4c18-8d9f-2ba62bc402ec None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.601373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 569/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:24 2022] PUT /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a/users/0002c704d39644778da298532e4f9ba2/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:56:26.582781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b2406-9d31-4404-8de9-b7b8e3c7a981 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.583289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b2406-9d31-4404-8de9-b7b8e3c7a981 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.583374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b2406-9d31-4404-8de9-b7b8e3c7a981 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.584714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 555/1124] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:26.590242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabea32-54d4-40eb-b247-47fdeeb6571c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.590648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabea32-54d4-40eb-b247-47fdeeb6571c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.590741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabea32-54d4-40eb-b247-47fdeeb6571c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.608955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7fabea32-54d4-40eb-b247-47fdeeb6571c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:26.610139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7fabea32-54d4-40eb-b247-47fdeeb6571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:26.642773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:26.675189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 570/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:26.697932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b2406-9d31-4404-8de9-b7b8e3c7a981 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:26.710669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:26.799595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b2406-9d31-4404-8de9-b7b8e3c7a981 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:26.831058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:26.831673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.831949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.832219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] PATH_INFO: `/v3/roles/d5d74a92228c4f3cabde0ad429390b8a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.833651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] RBAC: Authorizing `identity:get_role(role_id=d5d74a92228c4f3cabde0ad429390b8a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:26.834992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:26.836832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 556/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:26 2022] GET /identity/v3/roles/d5d74a92228c4f3cabde0ad429390b8a => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:26.842695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabea32-54d4-40eb-b247-47fdeeb6571c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:26.938329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabea32-54d4-40eb-b247-47fdeeb6571c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:26.971723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:26.972353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.972667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.972944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] PATH_INFO: `/v3/users/0002c704d39644778da298532e4f9ba2` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.974104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] RBAC: Authorizing `identity:get_user(user_id=0002c704d39644778da298532e4f9ba2)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:26.975531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:26.977183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 571/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:26 2022] GET /identity/v3/users/0002c704d39644778da298532e4f9ba2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:26.982895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.054220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:27.077973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018d483-44d5-4ed3-a302-00e42c3504e7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:27.110901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:27.111433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.111707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.111974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.175475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:27.176912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:27.245464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:27.269423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 557/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:27.275969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.315411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:27.373582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6c81-33ca-4afa-8eb7-fd5b68788be3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:27.406428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f669ab-4da4-4330-849c-9f35e18014fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:27.407046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f669ab-4da4-4330-849c-9f35e18014fa None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.407321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f669ab-4da4-4330-849c-9f35e18014fa None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.407590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f669ab-4da4-4330-849c-9f35e18014fa None admin] PATH_INFO: `/v3/projects/262b85d39a3842e8aeb0ba1424ba632a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.408747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f669ab-4da4-4330-849c-9f35e18014fa None admin] RBAC: Authorizing `identity:get_project(project_id=262b85d39a3842e8aeb0ba1424ba632a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:27.410165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f669ab-4da4-4330-849c-9f35e18014fa None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:27.411800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 572/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:27 2022] GET /identity/v3/projects/262b85d39a3842e8aeb0ba1424ba632a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:27.417936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.516359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37175c6d-0f9c-4a95-b69b-f0be03ba00e2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:27.549253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770397a-edff-4f94-81b1-b3ced9d20aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:27.549855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770397a-edff-4f94-81b1-b3ced9d20aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.550133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770397a-edff-4f94-81b1-b3ced9d20aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.550405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770397a-edff-4f94-81b1-b3ced9d20aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.551182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770397a-edff-4f94-81b1-b3ced9d20aa6 None admin] RBAC: Adding query filter params (role.id=d5d74a92228c4f3cabde0ad429390b8a, scope.project.id=262b85d39a3842e8aeb0ba1424ba632a, user.id=0002c704d39644778da298532e4f9ba2) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:27.551489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770397a-edff-4f94-81b1-b3ced9d20aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:27.552936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770397a-edff-4f94-81b1-b3ced9d20aa6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:27.559048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 558/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:27 2022] GET /identity/v3/role_assignments?user.id=0002c704d39644778da298532e4f9ba2&scope.project.id=262b85d39a3842e8aeb0ba1424ba632a&role.id=d5d74a92228c4f3cabde0ad429390b8a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:29.562251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fcdb4-5a75-47f9-9c5d-7095cbaee4b6 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.562809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fcdb4-5a75-47f9-9c5d-7095cbaee4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.562809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fcdb4-5a75-47f9-9c5d-7095cbaee4b6 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.564207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 573/1131] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:29.569509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b42fd8-e1a3-4c78-ace3-ce2a8e7374a4 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.569926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b42fd8-e1a3-4c78-ace3-ce2a8e7374a4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.570064 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b42fd8-e1a3-4c78-ace3-ce2a8e7374a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.588141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b7b42fd8-e1a3-4c78-ace3-ce2a8e7374a4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:29.589412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b7b42fd8-e1a3-4c78-ace3-ce2a8e7374a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:29.618090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:29.654013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 559/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:29.677053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fcdb4-5a75-47f9-9c5d-7095cbaee4b6 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.775394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fcdb4-5a75-47f9-9c5d-7095cbaee4b6 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.782618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:29.809726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55321e8-7c3d-475c-a11f-2fe6525bcb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.810345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55321e8-7c3d-475c-a11f-2fe6525bcb42 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.810625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55321e8-7c3d-475c-a11f-2fe6525bcb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.810900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55321e8-7c3d-475c-a11f-2fe6525bcb42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.812085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55321e8-7c3d-475c-a11f-2fe6525bcb42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.814010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55321e8-7c3d-475c-a11f-2fe6525bcb42 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:29.815899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 574/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:29.821494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b42fd8-e1a3-4c78-ace3-ce2a8e7374a4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.922636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b42fd8-e1a3-4c78-ace3-ce2a8e7374a4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.955602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cbed1-4656-43c5-b435-08b5c3196afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.956212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cbed1-4656-43c5-b435-08b5c3196afc None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.956519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cbed1-4656-43c5-b435-08b5c3196afc None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.956794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cbed1-4656-43c5-b435-08b5c3196afc None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.957478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26cbed1-4656-43c5-b435-08b5c3196afc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.960578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26cbed1-4656-43c5-b435-08b5c3196afc None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:30.065290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 560/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:29 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:56:32.071565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311c2c-7b0a-4f37-9767-ad835bc55018 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.072207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311c2c-7b0a-4f37-9767-ad835bc55018 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.072207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311c2c-7b0a-4f37-9767-ad835bc55018 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.073475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 575/1135] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:32.078553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4842d-8260-4dd8-a18e-d31b12a1920c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.078986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4842d-8260-4dd8-a18e-d31b12a1920c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.079109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4842d-8260-4dd8-a18e-d31b12a1920c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.096841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-43f4842d-8260-4dd8-a18e-d31b12a1920c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:32.098026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-43f4842d-8260-4dd8-a18e-d31b12a1920c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:32.111666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.164412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 561/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:32.189216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a311c2c-7b0a-4f37-9767-ad835bc55018 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.287509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a311c2c-7b0a-4f37-9767-ad835bc55018 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.319960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28f417-5ccd-4f4e-a663-d9307a679216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.320600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28f417-5ccd-4f4e-a663-d9307a679216 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.320886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28f417-5ccd-4f4e-a663-d9307a679216 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.321158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28f417-5ccd-4f4e-a663-d9307a679216 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.322348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28f417-5ccd-4f4e-a663-d9307a679216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.323728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28f417-5ccd-4f4e-a663-d9307a679216 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.325328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 576/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:32.331097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4842d-8260-4dd8-a18e-d31b12a1920c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.430989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4842d-8260-4dd8-a18e-d31b12a1920c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.462983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04466001-d465-4f65-a357-2e95a62507d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.463596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466001-d465-4f65-a357-2e95a62507d5 None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.463882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466001-d465-4f65-a357-2e95a62507d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.464182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466001-d465-4f65-a357-2e95a62507d5 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.464899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04466001-d465-4f65-a357-2e95a62507d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.466318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04466001-d465-4f65-a357-2e95a62507d5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.566307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 562/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:56:32 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:34.595584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b330184-280c-4754-9703-7c8338b7a8d4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:34.596107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b330184-280c-4754-9703-7c8338b7a8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:34.596107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b330184-280c-4754-9703-7c8338b7a8d4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:34.597446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 577/1139] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:34.602632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e443af4-09f1-4423-b559-f285b923a910 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:34.603058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e443af4-09f1-4423-b559-f285b923a910 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:34.603193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e443af4-09f1-4423-b559-f285b923a910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:34.615835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:34.621070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7e443af4-09f1-4423-b559-f285b923a910 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:34.622247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7e443af4-09f1-4423-b559-f285b923a910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:34.687950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 563/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:34.712738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b330184-280c-4754-9703-7c8338b7a8d4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:34.755921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:34.811541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b330184-280c-4754-9703-7c8338b7a8d4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:34.844239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:34.844898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:34.845183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:34.845460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:34.855391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:34.860245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:34.861408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:34.862624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 578/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:34.867863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e443af4-09f1-4423-b559-f285b923a910 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:34.963929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e443af4-09f1-4423-b559-f285b923a910 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:34.997538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:34.998143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:34.998423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:34.998695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:34.999403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:34.999705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:35.001089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:35.003201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:35.009937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 564/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:35.015593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:35.113521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6719ca-ea50-4ce6-a958-79b70bf65642 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:35.115463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:35.147160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:35.147788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:35.148117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:35.148624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] PATH_INFO: `/v3/users/ebe3e47f2312419ea3c45582dea544aa` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:35.162593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] RBAC: Authorizing `identity:get_user(user_id=ebe3e47f2312419ea3c45582dea544aa)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:35.164024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:35.165677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 579/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:35 2022] GET /identity/v3/users/ebe3e47f2312419ea3c45582dea544aa => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:35.171477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:35.267802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425923d-a5ae-4ab9-9cbf-9902ac5b7144 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:35.300569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:35.301874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:35.301874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:35.301874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:35.365459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:35.368822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:35.370263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:35.464667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 565/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:35.470668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:35.568275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8efb3-d9d4-4cb3-bd24-fad510a6a747 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:35.601793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6ce5d-c3a9-41ae-a31a-a48dd8f47c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:35.602416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6ce5d-c3a9-41ae-a31a-a48dd8f47c45 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:35.602697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6ce5d-c3a9-41ae-a31a-a48dd8f47c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:35.602971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6ce5d-c3a9-41ae-a31a-a48dd8f47c45 None admin] PATH_INFO: `/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:35.604125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6ce5d-c3a9-41ae-a31a-a48dd8f47c45 None admin] RBAC: Authorizing `identity:get_project(project_id=f0cbec7a733d48f29fe07fc3ccc86ee3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:35.605579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6ce5d-c3a9-41ae-a31a-a48dd8f47c45 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:35.607214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 580/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:35 2022] GET /identity/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:35.613118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:35.704941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:35.709100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebadf024-ffd5-426e-a8ce-79f309371cce None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:35.741817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb2e26-fc3f-4ae1-a674-5c67c26068a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:35.742659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2e26-fc3f-4ae1-a674-5c67c26068a8 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:35.743120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2e26-fc3f-4ae1-a674-5c67c26068a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:35.743715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb2e26-fc3f-4ae1-a674-5c67c26068a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:35.744604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb2e26-fc3f-4ae1-a674-5c67c26068a8 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=f0cbec7a733d48f29fe07fc3ccc86ee3, user.id=ebe3e47f2312419ea3c45582dea544aa) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:35.744920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb2e26-fc3f-4ae1-a674-5c67c26068a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:35.746382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb2e26-fc3f-4ae1-a674-5c67c26068a8 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:35.751637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 566/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:35 2022] GET /identity/v3/role_assignments?user.id=ebe3e47f2312419ea3c45582dea544aa&scope.project.id=f0cbec7a733d48f29fe07fc3ccc86ee3&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:37.704873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24477bcd-cd70-476a-97a2-9c6d0a5beed2 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.704873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24477bcd-cd70-476a-97a2-9c6d0a5beed2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.704873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24477bcd-cd70-476a-97a2-9c6d0a5beed2 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.704873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 581/1147] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:37.706113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a11dc-be4b-4d7f-871b-f78b0a48e917 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.706553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a11dc-be4b-4d7f-871b-f78b0a48e917 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.706649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a11dc-be4b-4d7f-871b-f78b0a48e917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.724896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-425a11dc-be4b-4d7f-871b-f78b0a48e917 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:37.726077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-425a11dc-be4b-4d7f-871b-f78b0a48e917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:37.790503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 567/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:37.812789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24477bcd-cd70-476a-97a2-9c6d0a5beed2 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:37.840237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:37.911704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24477bcd-cd70-476a-97a2-9c6d0a5beed2 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:37.944673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:37.945298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.945581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.945855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.955709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:37.961278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:37.962314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:37.963154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 582/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:37.968467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a11dc-be4b-4d7f-871b-f78b0a48e917 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:38.057097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:38.065594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a11dc-be4b-4d7f-871b-f78b0a48e917 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:38.100011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:38.100648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:38.100936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:38.101209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:38.101917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:38.102223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:38.103577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:38.111393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 568/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:38.117294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:38.202302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:38.214413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b75744-5619-432e-a78a-4463d2e5e059 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:38.246869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:38.247496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:38.247781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:38.248060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] PATH_INFO: `/v3/users/ebe3e47f2312419ea3c45582dea544aa` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:38.249255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] RBAC: Authorizing `identity:get_user(user_id=ebe3e47f2312419ea3c45582dea544aa)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:38.250684 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:38.252282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 583/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:38 2022] GET /identity/v3/users/ebe3e47f2312419ea3c45582dea544aa => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:38.258452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:38.355454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac080067-3781-4296-b8d8-554e88d28903 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:38.387739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2053f8d-2989-4918-bcaa-efe06a5c00a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:38.388362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2053f8d-2989-4918-bcaa-efe06a5c00a0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:38.388668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2053f8d-2989-4918-bcaa-efe06a5c00a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:38.388938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2053f8d-2989-4918-bcaa-efe06a5c00a0 None admin] PATH_INFO: `/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:38.390062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2053f8d-2989-4918-bcaa-efe06a5c00a0 None admin] RBAC: Authorizing `identity:get_project(project_id=f0cbec7a733d48f29fe07fc3ccc86ee3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:38.391464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2053f8d-2989-4918-bcaa-efe06a5c00a0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:38.393113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 569/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:38 2022] GET /identity/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:38.399561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:38.494668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238b69d-8021-423b-9d1f-15a01b4295be None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:38.527696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cbec3-5fa3-49bf-912f-66e023581320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:38.528605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cbec3-5fa3-49bf-912f-66e023581320 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:38.528898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cbec3-5fa3-49bf-912f-66e023581320 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:38.529169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cbec3-5fa3-49bf-912f-66e023581320 None admin] PATH_INFO: `/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3/users/ebe3e47f2312419ea3c45582dea544aa/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:38.531090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cbec3-5fa3-49bf-912f-66e023581320 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0cbec7a733d48f29fe07fc3ccc86ee3, user_id=ebe3e47f2312419ea3c45582dea544aa, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:38.532530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cbec3-5fa3-49bf-912f-66e023581320 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:38.616048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 584/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:38 2022] PUT /identity/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3/users/ebe3e47f2312419ea3c45582dea544aa/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:56:40.607133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2952-41a4-417f-bf2b-4e3a9af2fd7b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.607643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2952-41a4-417f-bf2b-4e3a9af2fd7b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.611671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2952-41a4-417f-bf2b-4e3a9af2fd7b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.614413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 570/1154] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:40.619525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b737f3-d774-4be8-acf7-8bc517c0fe02 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.619948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b737f3-d774-4be8-acf7-8bc517c0fe02 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.620071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b737f3-d774-4be8-acf7-8bc517c0fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.637719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-81b737f3-d774-4be8-acf7-8bc517c0fe02 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:40.638900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-81b737f3-d774-4be8-acf7-8bc517c0fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:40.649220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:40.702990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 585/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:40.726375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2952-41a4-417f-bf2b-4e3a9af2fd7b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:40.750946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:40.825085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2952-41a4-417f-bf2b-4e3a9af2fd7b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:40.857445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:40.858007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.858285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.858556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.869705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:40.874635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:40.875690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:40.876558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 571/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:40.882412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b737f3-d774-4be8-acf7-8bc517c0fe02 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:40.977415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b737f3-d774-4be8-acf7-8bc517c0fe02 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:41.011513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:41.012120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:41.012424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:41.012706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:41.013420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:41.013724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:41.015079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:41.022890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 586/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:41.028787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:41.109110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:41.126523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab2248-0427-43e1-8e62-f9be7f8be1cb None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:41.160711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:41.161339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:41.161620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:41.161893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] PATH_INFO: `/v3/users/ebe3e47f2312419ea3c45582dea544aa` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:41.163073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] RBAC: Authorizing `identity:get_user(user_id=ebe3e47f2312419ea3c45582dea544aa)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:41.164530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:41.166138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 572/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:41 2022] GET /identity/v3/users/ebe3e47f2312419ea3c45582dea544aa => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:41.171930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:41.183711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:41.268777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941464f-04af-444e-9197-a33d35a3c9f1 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:41.301429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:41.302027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:41.302299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:41.302563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:41.367806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:41.369209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:41.378251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:41.461950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 587/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:41.467777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:41.564452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260e283-f32b-40b3-963b-865e4a1d5c81 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:41.596515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd4a3b-84fb-4d9c-8558-62ffde1dee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:41.597158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4a3b-84fb-4d9c-8558-62ffde1dee85 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:41.598012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4a3b-84fb-4d9c-8558-62ffde1dee85 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:41.598295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4a3b-84fb-4d9c-8558-62ffde1dee85 None admin] PATH_INFO: `/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:41.599453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd4a3b-84fb-4d9c-8558-62ffde1dee85 None admin] RBAC: Authorizing `identity:get_project(project_id=f0cbec7a733d48f29fe07fc3ccc86ee3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:41.600905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd4a3b-84fb-4d9c-8558-62ffde1dee85 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:41.602529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 573/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:41 2022] GET /identity/v3/projects/f0cbec7a733d48f29fe07fc3ccc86ee3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:41.608353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:41.706246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f409eb-41b7-4184-8bb7-9b31f8ad1975 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:41.715402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:41.739222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6ccd1-4697-4b19-a94e-12875abfa84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:41.739815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ccd1-4697-4b19-a94e-12875abfa84d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:41.740091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ccd1-4697-4b19-a94e-12875abfa84d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:41.740362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6ccd1-4697-4b19-a94e-12875abfa84d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:41.741237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6ccd1-4697-4b19-a94e-12875abfa84d None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=f0cbec7a733d48f29fe07fc3ccc86ee3, user.id=ebe3e47f2312419ea3c45582dea544aa) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:41.741806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6ccd1-4697-4b19-a94e-12875abfa84d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:41.743369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6ccd1-4697-4b19-a94e-12875abfa84d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:41.749468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 588/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:41 2022] GET /identity/v3/role_assignments?user.id=ebe3e47f2312419ea3c45582dea544aa&scope.project.id=f0cbec7a733d48f29fe07fc3ccc86ee3&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:43.755141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f2b95-306e-494e-8274-1c3b1e8ae347 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:43.755654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f2b95-306e-494e-8274-1c3b1e8ae347 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:43.755748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f2b95-306e-494e-8274-1c3b1e8ae347 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:43.757041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 574/1162] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:43.762313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9637ba-bd71-4906-8199-db625d82f501 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:43.762710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9637ba-bd71-4906-8199-db625d82f501 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:43.762886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9637ba-bd71-4906-8199-db625d82f501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:43.781407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ae9637ba-bd71-4906-8199-db625d82f501 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:43.782584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ae9637ba-bd71-4906-8199-db625d82f501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:43.848109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 589/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:43.869642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f2b95-306e-494e-8274-1c3b1e8ae347 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:43.888221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:43.969705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f2b95-306e-494e-8274-1c3b1e8ae347 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:44.004254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:44.004893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:44.005220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:44.005442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:44.070791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:44.072232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:44.089664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:44.168541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 575/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:44.174663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9637ba-bd71-4906-8199-db625d82f501 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:44.269610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9637ba-bd71-4906-8199-db625d82f501 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:44.271035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:44.302480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49961fad-c90a-4a2e-afa8-8edb27afc439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:44.303108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49961fad-c90a-4a2e-afa8-8edb27afc439 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:44.303390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49961fad-c90a-4a2e-afa8-8edb27afc439 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:44.303666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49961fad-c90a-4a2e-afa8-8edb27afc439 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:44.310188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49961fad-c90a-4a2e-afa8-8edb27afc439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:44.311629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49961fad-c90a-4a2e-afa8-8edb27afc439 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-49961fad-c90a-4a2e-afa8-8edb27afc439 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:44.324114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:44.325354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:56:44.326421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 590/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:56:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:44.331997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:44.426283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b9289-340a-46f1-94c2-22d744f2dba1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:44.443046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:44.459564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00571978-fa77-4e2f-a81d-fa4ee4fef9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:44.460448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571978-fa77-4e2f-a81d-fa4ee4fef9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:44.460560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571978-fa77-4e2f-a81d-fa4ee4fef9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:44.460748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00571978-fa77-4e2f-a81d-fa4ee4fef9a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:44.461437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00571978-fa77-4e2f-a81d-fa4ee4fef9a5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:44.461745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00571978-fa77-4e2f-a81d-fa4ee4fef9a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:44.463132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00571978-fa77-4e2f-a81d-fa4ee4fef9a5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:44.470044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 576/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:46.447693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17dd2-cbdf-4d15-bbf4-fb3f8dafd09b None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:46.448216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17dd2-cbdf-4d15-bbf4-fb3f8dafd09b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:46.448216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c17dd2-cbdf-4d15-bbf4-fb3f8dafd09b None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:46.449633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 591/1167] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:46.454788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c2c9-66ec-4462-8595-be3dacb98b77 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:46.455203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c2c9-66ec-4462-8595-be3dacb98b77 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:46.455341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c2c9-66ec-4462-8595-be3dacb98b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:46.473668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-0874c2c9-66ec-4462-8595-be3dacb98b77 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:46.474861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0874c2c9-66ec-4462-8595-be3dacb98b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:46.542993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 577/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:46.565620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c17dd2-cbdf-4d15-bbf4-fb3f8dafd09b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:46.663547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c17dd2-cbdf-4d15-bbf4-fb3f8dafd09b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:46.695925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89feb2-144f-4bb3-8b1f-51f1f174fac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:46.696553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89feb2-144f-4bb3-8b1f-51f1f174fac3 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:46.696835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89feb2-144f-4bb3-8b1f-51f1f174fac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:46.697109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89feb2-144f-4bb3-8b1f-51f1f174fac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:46.697817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89feb2-144f-4bb3-8b1f-51f1f174fac3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:46.699173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89feb2-144f-4bb3-8b1f-51f1f174fac3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:46.798800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 592/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Wed Jul 27 19:56:46 2022] POST /identity/v3/domains => generated 267 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:48.809146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d892a4-c526-4887-9b5d-d17271ea1852 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:48.809735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d892a4-c526-4887-9b5d-d17271ea1852 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:48.809735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d892a4-c526-4887-9b5d-d17271ea1852 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:48.811016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 578/1170] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:48.816570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8873c5-3ad3-4731-8a64-97bd76868a23 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:48.817011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8873c5-3ad3-4731-8a64-97bd76868a23 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:48.817126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8873c5-3ad3-4731-8a64-97bd76868a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:48.835403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6e8873c5-3ad3-4731-8a64-97bd76868a23 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:48.836616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6e8873c5-3ad3-4731-8a64-97bd76868a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:48.852538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:48.901370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 593/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:48.924524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d892a4-c526-4887-9b5d-d17271ea1852 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:49.021977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d892a4-c526-4887-9b5d-d17271ea1852 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:49.035286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:49.058476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0edf3-10f3-4a0c-a6a9-0c1c6280fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:49.060578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0edf3-10f3-4a0c-a6a9-0c1c6280fb61 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:49.060819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0edf3-10f3-4a0c-a6a9-0c1c6280fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:49.061100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0edf3-10f3-4a0c-a6a9-0c1c6280fb61 None admin] PATH_INFO: `/v3/domains/026df896e3ba48f29db4f15d26fecfae` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:49.081042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0edf3-10f3-4a0c-a6a9-0c1c6280fb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=026df896e3ba48f29db4f15d26fecfae)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:49.084370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0edf3-10f3-4a0c-a6a9-0c1c6280fb61 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:49.086074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 579/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Jul 27 19:56:48 2022] GET /identity/v3/domains/026df896e3ba48f29db4f15d26fecfae => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:49.091938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8873c5-3ad3-4731-8a64-97bd76868a23 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:49.187410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8873c5-3ad3-4731-8a64-97bd76868a23 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:49.222382 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31022b0d-4625-4290-9595-f45e687c2f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:49.222996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31022b0d-4625-4290-9595-f45e687c2f29 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:49.223276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31022b0d-4625-4290-9595-f45e687c2f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:49.223552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31022b0d-4625-4290-9595-f45e687c2f29 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:49.224247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31022b0d-4625-4290-9595-f45e687c2f29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:49.225708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31022b0d-4625-4290-9595-f45e687c2f29 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:49.312808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 594/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:49 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:51.318807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea298ce-512a-4118-bf2f-68b49c891c41 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.319365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea298ce-512a-4118-bf2f-68b49c891c41 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.319365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea298ce-512a-4118-bf2f-68b49c891c41 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.320705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 580/1174] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:51.325828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926458d9-4f62-4a49-949f-cbc74e768f4c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.326253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926458d9-4f62-4a49-949f-cbc74e768f4c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.326358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926458d9-4f62-4a49-949f-cbc74e768f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.333557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:51.344238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-926458d9-4f62-4a49-949f-cbc74e768f4c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:51.345453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-926458d9-4f62-4a49-949f-cbc74e768f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:51.408164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 595/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:51.432192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea298ce-512a-4118-bf2f-68b49c891c41 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.531162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea298ce-512a-4118-bf2f-68b49c891c41 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:51.563052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30196e5-ae48-40a4-b354-662b37938dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:51.563685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30196e5-ae48-40a4-b354-662b37938dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.564603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30196e5-ae48-40a4-b354-662b37938dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.564889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30196e5-ae48-40a4-b354-662b37938dd5 None admin] PATH_INFO: `/v3/domains/026df896e3ba48f29db4f15d26fecfae` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.566038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30196e5-ae48-40a4-b354-662b37938dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=026df896e3ba48f29db4f15d26fecfae)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:51.567489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30196e5-ae48-40a4-b354-662b37938dd5 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:51.569175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 581/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Jul 27 19:56:51 2022] GET /identity/v3/domains/026df896e3ba48f29db4f15d26fecfae => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:51.574817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926458d9-4f62-4a49-949f-cbc74e768f4c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.669012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926458d9-4f62-4a49-949f-cbc74e768f4c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:51.701276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff567ff-5f92-41ca-a44a-0aab6e40108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:51.701867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff567ff-5f92-41ca-a44a-0aab6e40108f None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.702167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff567ff-5f92-41ca-a44a-0aab6e40108f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.702412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff567ff-5f92-41ca-a44a-0aab6e40108f None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.703094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff567ff-5f92-41ca-a44a-0aab6e40108f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:51.704574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff567ff-5f92-41ca-a44a-0aab6e40108f None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:51.712258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:51.803192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 596/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:56:51 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:53.826762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b26d5a-bc0e-4015-bf9f-a18df295f19b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:53.827304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b26d5a-bc0e-4015-bf9f-a18df295f19b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:53.827304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b26d5a-bc0e-4015-bf9f-a18df295f19b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:53.828680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 582/1178] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:53.833865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76851d9d-99bd-4d38-b61d-8b39faa53f62 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:53.834283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76851d9d-99bd-4d38-b61d-8b39faa53f62 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:53.834427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76851d9d-99bd-4d38-b61d-8b39faa53f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:53.852171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-76851d9d-99bd-4d38-b61d-8b39faa53f62 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:53.853397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-76851d9d-99bd-4d38-b61d-8b39faa53f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:53.918892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 597/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:53.942364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b26d5a-bc0e-4015-bf9f-a18df295f19b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:53.983665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.041095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b26d5a-bc0e-4015-bf9f-a18df295f19b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.073170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.073790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.074069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.074340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.088954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:54.094013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:54.095693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:54.096986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 583/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:54.101756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76851d9d-99bd-4d38-b61d-8b39faa53f62 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.201873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76851d9d-99bd-4d38-b61d-8b39faa53f62 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.221218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.234268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.234880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.235161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.235984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.236727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:54.237033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.238392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.246094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 598/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:54.251912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.348124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.351074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a33b1-977e-45c0-a127-3b1028986ea2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.383321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.383946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.384226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.384531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] PATH_INFO: `/v3/users/28154d1016fb458b9f7122022c202bc0` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.397563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] RBAC: Authorizing `identity:get_user(user_id=28154d1016fb458b9f7122022c202bc0)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.398979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.400846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 584/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/users/28154d1016fb458b9f7122022c202bc0 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:54.406551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.502464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59215be6-dfd2-48e5-b661-6c2940f94070 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.535154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.535755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.536035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.536309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.582309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.600175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.601589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.694852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 599/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:54.700714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.799946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92631489-c6f6-4856-94ab-a318b2eadb99 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.832801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4d2d-8800-481d-ae60-3233c9a15ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.833440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4d2d-8800-481d-ae60-3233c9a15ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.833693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4d2d-8800-481d-ae60-3233c9a15ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.833962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4d2d-8800-481d-ae60-3233c9a15ac7 None admin] PATH_INFO: `/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.835102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4d2d-8800-481d-ae60-3233c9a15ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=dc0958cb4fd54e7bae9db8ba853c2d65)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.836541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4d2d-8800-481d-ae60-3233c9a15ac7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.838176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 585/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:54.844341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.921875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.939182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b495a-0b03-43b5-a189-3fadf8653140 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.971626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186eae4-60ee-4473-8874-101ca422bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.972223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186eae4-60ee-4473-8874-101ca422bef2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.972527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186eae4-60ee-4473-8874-101ca422bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.972799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186eae4-60ee-4473-8874-101ca422bef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.973576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186eae4-60ee-4473-8874-101ca422bef2 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=dc0958cb4fd54e7bae9db8ba853c2d65, user.id=28154d1016fb458b9f7122022c202bc0) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:54.973881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186eae4-60ee-4473-8874-101ca422bef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.975284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186eae4-60ee-4473-8874-101ca422bef2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.980445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 600/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/role_assignments?user.id=28154d1016fb458b9f7122022c202bc0&scope.project.id=dc0958cb4fd54e7bae9db8ba853c2d65&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:56.932941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4e06-5294-4640-a9a9-7e2093b80926 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:56.933496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4e06-5294-4640-a9a9-7e2093b80926 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:56.933496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4e06-5294-4640-a9a9-7e2093b80926 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:56.934814 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 586/1186] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:56.940419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315628e-9eb8-4510-af76-f6b16cecc49b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:56.940934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315628e-9eb8-4510-af76-f6b16cecc49b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:56.941087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315628e-9eb8-4510-af76-f6b16cecc49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:56.959795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1315628e-9eb8-4510-af76-f6b16cecc49b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:56.961023 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1315628e-9eb8-4510-af76-f6b16cecc49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:57.025319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 601/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:57.051651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4e06-5294-4640-a9a9-7e2093b80926 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.079138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.148906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4e06-5294-4640-a9a9-7e2093b80926 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.180604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.181226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.181503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.181772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.192160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:57.196971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:57.198039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:56:57.198981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 587/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:57.204604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315628e-9eb8-4510-af76-f6b16cecc49b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.278087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.300973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315628e-9eb8-4510-af76-f6b16cecc49b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.334454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.335083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.335362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.335637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.336364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:57.336692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.338063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.346515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 602/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:57.352435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.434593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.449417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2c03f-e9c2-4712-9f0f-b53ea0ad6403 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.481627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.482248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.482527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.482802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] PATH_INFO: `/v3/users/28154d1016fb458b9f7122022c202bc0` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.483944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] RBAC: Authorizing `identity:get_user(user_id=28154d1016fb458b9f7122022c202bc0)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.485868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.487435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 588/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/users/28154d1016fb458b9f7122022c202bc0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:57.493058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.589250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334aee69-fbea-41b2-b26d-f693014137d7 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.621320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634db2ed-acde-414f-8330-f05779811949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.621940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db2ed-acde-414f-8330-f05779811949 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.622218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db2ed-acde-414f-8330-f05779811949 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.622492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db2ed-acde-414f-8330-f05779811949 None admin] PATH_INFO: `/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.623650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634db2ed-acde-414f-8330-f05779811949 None admin] RBAC: Authorizing `identity:get_project(project_id=dc0958cb4fd54e7bae9db8ba853c2d65)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.625680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634db2ed-acde-414f-8330-f05779811949 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.627312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 603/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:57.633147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.729329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde60799-e3da-4961-8b59-d3573b909550 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.761971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129522e-8039-47a2-8cc1-56bacf7185af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.762605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129522e-8039-47a2-8cc1-56bacf7185af None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.762883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129522e-8039-47a2-8cc1-56bacf7185af None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.763157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129522e-8039-47a2-8cc1-56bacf7185af None admin] PATH_INFO: `/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65/users/28154d1016fb458b9f7122022c202bc0/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.765375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129522e-8039-47a2-8cc1-56bacf7185af None admin] RBAC: Authorizing `identity:create_grant(project_id=dc0958cb4fd54e7bae9db8ba853c2d65, user_id=28154d1016fb458b9f7122022c202bc0, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.766812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129522e-8039-47a2-8cc1-56bacf7185af None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.868742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 589/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:57 2022] PUT /identity/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65/users/28154d1016fb458b9f7122022c202bc0/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:56:59.819407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc4443-4b23-4cb0-bfa0-41cd552a6e25 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:59.819939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc4443-4b23-4cb0-bfa0-41cd552a6e25 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:59.819939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc4443-4b23-4cb0-bfa0-41cd552a6e25 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:59.821318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 604/1193] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:59.826332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fea96f-239e-47ef-b821-5ca77d811c55 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:59.826760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fea96f-239e-47ef-b821-5ca77d811c55 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:59.826894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fea96f-239e-47ef-b821-5ca77d811c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:59.845672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-59fea96f-239e-47ef-b821-5ca77d811c55 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:59.846851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-59fea96f-239e-47ef-b821-5ca77d811c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:59.855832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:59.911494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 590/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:59.934074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc4443-4b23-4cb0-bfa0-41cd552a6e25 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:59.943713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.034447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc4443-4b23-4cb0-bfa0-41cd552a6e25 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.067452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.068071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.068354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.068656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.079720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:57:00.084608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:57:00.085751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:57:00.086773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 605/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:57:00.091668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fea96f-239e-47ef-b821-5ca77d811c55 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.188827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fea96f-239e-47ef-b821-5ca77d811c55 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.222022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.222626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.222903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.223170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.223870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:00.224168 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.225617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.233387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 591/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:00.239294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.302834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.336668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c284f4-a3b7-467d-adb9-5df55ab59663 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.369445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.370075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.370356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.370634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] PATH_INFO: `/v3/users/28154d1016fb458b9f7122022c202bc0` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.371778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] RBAC: Authorizing `identity:get_user(user_id=28154d1016fb458b9f7122022c202bc0)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.373248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.375501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 606/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/users/28154d1016fb458b9f7122022c202bc0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:00.381487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.390657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.478863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae373f9-b75b-4e9a-b8a6-6ae38a13b3a4 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.511317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.511924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.512206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.512576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.576438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.577895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.585106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.672486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 592/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:57:00.678351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.775596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa61bbf-8a4a-4a01-83f6-08b9db19a7a3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.807612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a249f-585c-479a-8e21-2814859f3984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.808239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a249f-585c-479a-8e21-2814859f3984 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.808557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a249f-585c-479a-8e21-2814859f3984 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.808836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a249f-585c-479a-8e21-2814859f3984 None admin] PATH_INFO: `/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.809987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a249f-585c-479a-8e21-2814859f3984 None admin] RBAC: Authorizing `identity:get_project(project_id=dc0958cb4fd54e7bae9db8ba853c2d65)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.811434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a249f-585c-479a-8e21-2814859f3984 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.813629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 607/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/projects/dc0958cb4fd54e7bae9db8ba853c2d65 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:00.819640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.914901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd522bf3-989c-47d6-b718-f3d62fc2326c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.922767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.948699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b42b79-62fd-49f1-bfda-c33171bef5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.949298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42b79-62fd-49f1-bfda-c33171bef5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.949578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42b79-62fd-49f1-bfda-c33171bef5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.949851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42b79-62fd-49f1-bfda-c33171bef5c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.950628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b42b79-62fd-49f1-bfda-c33171bef5c1 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=dc0958cb4fd54e7bae9db8ba853c2d65, user.id=28154d1016fb458b9f7122022c202bc0) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:00.950938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b42b79-62fd-49f1-bfda-c33171bef5c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.952352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b42b79-62fd-49f1-bfda-c33171bef5c1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.959589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 593/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/role_assignments?user.id=28154d1016fb458b9f7122022c202bc0&scope.project.id=dc0958cb4fd54e7bae9db8ba853c2d65&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:10.803600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:10.960518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=15 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:10.858990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d75a6-0308-4a9b-ab33-fa65738218b4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:10.859606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d75a6-0308-4a9b-ab33-fa65738218b4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:10.859606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d75a6-0308-4a9b-ab33-fa65738218b4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:10.861586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 608/1201] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:10.866843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f899ed-e0eb-4a92-87f1-1de53d94db66 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:10.870021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f899ed-e0eb-4a92-87f1-1de53d94db66 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:10.870021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f899ed-e0eb-4a92-87f1-1de53d94db66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:10.885346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-39f899ed-e0eb-4a92-87f1-1de53d94db66 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:10.886821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-39f899ed-e0eb-4a92-87f1-1de53d94db66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:10.953373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 594/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:10.978495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d75a6-0308-4a9b-ab33-fa65738218b4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:11.079785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d75a6-0308-4a9b-ab33-fa65738218b4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:11.112875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:11.113441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:11.113719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:11.113990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:11.124144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:11.129368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:11.139583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:11.144489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:11.145935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:11.145935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:11.145935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:11.145935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:11.145935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 609/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:11.147198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f899ed-e0eb-4a92-87f1-1de53d94db66 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:11.243710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f899ed-e0eb-4a92-87f1-1de53d94db66 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:11.271837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:11.279733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cfac1-15bf-4a24-a071-58c09d038364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:11.279733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cfac1-15bf-4a24-a071-58c09d038364 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:11.279733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cfac1-15bf-4a24-a071-58c09d038364 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:11.279733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cfac1-15bf-4a24-a071-58c09d038364 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:11.280114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cfac1-15bf-4a24-a071-58c09d038364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:11.280114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cfac1-15bf-4a24-a071-58c09d038364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:11.281042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cfac1-15bf-4a24-a071-58c09d038364 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:11.292518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 595/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:11.298114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:11.350976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:11.396181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd85b7c-aade-4883-9237-5f6a39a28a01 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:11.432156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8b617-aa18-4882-865b-792e370f98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:11.432798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8b617-aa18-4882-865b-792e370f98c3 None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:11.433072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8b617-aa18-4882-865b-792e370f98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:11.433337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8b617-aa18-4882-865b-792e370f98c3 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:11.434012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8b617-aa18-4882-865b-792e370f98c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:11.435420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8b617-aa18-4882-865b-792e370f98c3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:11.518842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 610/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Jul 27 19:58:11 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:58:13.548037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5b7ea-9e87-4316-9302-2388f2de6ff8 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:13.548602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5b7ea-9e87-4316-9302-2388f2de6ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:13.548602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5b7ea-9e87-4316-9302-2388f2de6ff8 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:13.549968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 596/1206] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:13.555014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee590440-c53a-4e55-9197-85e32035f765 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:13.555448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee590440-c53a-4e55-9197-85e32035f765 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:13.555545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee590440-c53a-4e55-9197-85e32035f765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:13.573270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ee590440-c53a-4e55-9197-85e32035f765 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:13.574441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ee590440-c53a-4e55-9197-85e32035f765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:13.642851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 611/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:13.665849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5b7ea-9e87-4316-9302-2388f2de6ff8 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:13.766042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5b7ea-9e87-4316-9302-2388f2de6ff8 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:13.798627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:13.799255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:13.799536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:13.799808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:13.810025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:13.815561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:13.816723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:13.817627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 597/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:58:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:13.822695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee590440-c53a-4e55-9197-85e32035f765 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:13.862386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:13.920316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee590440-c53a-4e55-9197-85e32035f765 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:13.952556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:13.953819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:13.954097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:13.954367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:13.955079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:13.955382 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:13.956771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:13.964352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 612/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:58:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:13.970302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:14.028285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:14.078556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01452aa5-1257-4aeb-9d41-1bf9a20b5d57 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:14.111186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:14.111830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:14.112110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:14.112414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:14.122822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:14.124313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:14.140196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:14.141504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:14.142545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:14.142545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:14.142545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:14.142545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:14.142545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 598/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:14.147416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:14.243469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:14.246286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd0445-b7bc-4ed4-88e1-2d1fbc2c6133 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:14.279336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:14.279940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:14.280217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:14.280515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:14.281244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:14.281543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:14.282906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:14.294076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 613/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:14.300420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:14.399727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181fa9a-d5da-4d46-9e61-9e37b731b767 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:14.401275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:14.432028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:14.432750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:14.433028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:14.433297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:14.441155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:14.442619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:14.453947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:14.455142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:14.456268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 599/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:14.461258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:14.558887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c73fa-1e98-4349-8787-2ce4b9f742e1 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:14.592151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:14.592787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:14.593065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:14.593335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:14.594647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:14.594951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:14.596429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:14.613370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 614/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:14.619270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:14.716335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19480076-52d2-47cf-ae25-a2a064d04ee0 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:14.749364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:14.750049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:14.750300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:14.750568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:14.779153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:14.816344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:14.817750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:14.916903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 600/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:14.922712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:14.941379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:15.019341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a99213-5e6f-4da1-9875-1c501a0f24b7 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:15.053561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:15.054197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.054474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.054742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.064643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:15.066145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:15.080899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:15.081998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:15.083183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:15.083183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:15.083183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:15.083183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:15.083183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 615/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:15.087863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:15.148777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:15.186743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffec98-ef9f-4bff-a9ac-ea6f5a620f7f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:15.221162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:15.221775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.222058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.222335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.223023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:15.223443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:15.224843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:15.236040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 601/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:15.241723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:15.309508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:15.340097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3080ab-eefa-4696-a1b6-5656051985d2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:15.371884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:15.372543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.372824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.373099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.380981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:15.382425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:15.397317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:15.398431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:15.399607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:15.403156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 616/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:58:15.404217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:15.501231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1aea-caef-4525-b391-40519df46018 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:15.525494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:15.536954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b855b15-081f-4423-96bd-469b17363cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:15.537575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855b15-081f-4423-96bd-469b17363cac None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.537853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855b15-081f-4423-96bd-469b17363cac None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.538127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855b15-081f-4423-96bd-469b17363cac None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.538924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b855b15-081f-4423-96bd-469b17363cac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:15.539231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b855b15-081f-4423-96bd-469b17363cac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:15.540688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b855b15-081f-4423-96bd-469b17363cac None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:15.552196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 602/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:15.558415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:15.657901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a834-2be5-4d96-96e2-dfb16b6b03bb None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:15.685083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:15.691320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b370c-9b9f-4221-8572-f7f9f4cfc385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:15.694861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b370c-9b9f-4221-8572-f7f9f4cfc385 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.694861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b370c-9b9f-4221-8572-f7f9f4cfc385 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.694861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b370c-9b9f-4221-8572-f7f9f4cfc385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.694861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b370c-9b9f-4221-8572-f7f9f4cfc385 None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=328ec1aac43349b28a72d5df19fd2cdd) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:15.694861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b370c-9b9f-4221-8572-f7f9f4cfc385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:15.694861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b370c-9b9f-4221-8572-f7f9f4cfc385 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:15.700131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 617/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:15 2022] GET /identity/v3/role_assignments?user.id=328ec1aac43349b28a72d5df19fd2cdd&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:17.668901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1d6dd-3004-43eb-8d2a-73cab992644b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:17.669398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1d6dd-3004-43eb-8d2a-73cab992644b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:17.669398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1d6dd-3004-43eb-8d2a-73cab992644b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.670706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 603/1220] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:17.675804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c5ea2-97ef-42a4-b8bc-715f32ba2632 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:17.676230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c5ea2-97ef-42a4-b8bc-715f32ba2632 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:17.676334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c5ea2-97ef-42a4-b8bc-715f32ba2632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.695122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-345c5ea2-97ef-42a4-b8bc-715f32ba2632 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:17.696306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-345c5ea2-97ef-42a4-b8bc-715f32ba2632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:17.763084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 618/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:17.785971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1d6dd-3004-43eb-8d2a-73cab992644b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:17.887950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1d6dd-3004-43eb-8d2a-73cab992644b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:17.920725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:17.921785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:17.922068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:17.922339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.932295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:17.937763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:17.938798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:17.939907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 604/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:17.945021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c5ea2-97ef-42a4-b8bc-715f32ba2632 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:18.044639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c5ea2-97ef-42a4-b8bc-715f32ba2632 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:18.067599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:18.078523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:18.079139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:18.079456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:18.079740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:18.080519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] RBAC: Adding query filter params (name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:18.080824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:18.082212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:18.090003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 619/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:58:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:18.096553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:18.139110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:18.192758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da4c32-8677-4f6f-8973-6c2b4393294e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:18.225488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:18.226106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:18.226382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:18.226651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:18.236528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:18.237952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:18.252804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:18.253859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:18.254822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:18.254822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:18.254822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:18.254822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:18.254822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 605/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:18.259905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:18.359367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39d5b1-965c-4114-8e42-b460d5cd0879 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:18.393385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:18.393997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:18.394277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:18.394592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:18.395403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:18.395707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:18.397098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:18.408680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 620/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:18.414339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:18.506960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:18.513772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de89bcb-d3cf-4c0c-9ff9-206ec1d48c6c None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:18.546442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:18.547073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:18.547353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:18.547628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:18.554940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:18.556495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:18.568685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:18.570214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:18.571261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 606/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:18.575868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:18.622785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:18.677055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a86fe5-4274-4b04-8c29-3097d8255917 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:18.710411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:18.711015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:18.711294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:18.711565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:18.712358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:18.712691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:18.714101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:18.730566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 621/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:18.736868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:18.836823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde8553-e86b-4179-be94-b4e3c9bdcb76 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:18.869875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:18.870505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:18.870786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:18.871060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:18.882374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:18.883692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:18.898927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:18.899958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:18.900973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:18.900973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:18.900973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:18.900973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:18.900973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 607/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:18.905839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:19.002936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46821022-2c7d-43e4-99a4-e8511c44cbf7 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:19.005381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:19.036488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:19.037099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:19.037377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:19.037648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:19.038337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:19.038697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:19.040058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:19.051492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 622/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:19.057456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:19.067572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:19.153738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7c57-138f-4fb8-99e9-eb32cd23298b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:19.186116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:19.186742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:19.187042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:19.187310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:19.193726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:19.195174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:19.206174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:19.212048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:19.213142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 608/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:19.213729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:19.315549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03108600-5411-42d1-bc24-d11272d5a08a None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:19.350397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27033aa8-7749-451a-9ddc-9d757b890fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:19.351011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033aa8-7749-451a-9ddc-9d757b890fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:19.351293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033aa8-7749-451a-9ddc-9d757b890fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:19.351569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033aa8-7749-451a-9ddc-9d757b890fe9 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:19.352319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27033aa8-7749-451a-9ddc-9d757b890fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:19.352658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27033aa8-7749-451a-9ddc-9d757b890fe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:19.354084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27033aa8-7749-451a-9ddc-9d757b890fe9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:19.366649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 623/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:19.374873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:19.437868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:19.472711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad21bb-bad0-4511-923f-8237cfaf1e9d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:19.504386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc1363-f1b5-4ffa-a513-d6d8cff16d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:19.505061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1363-f1b5-4ffa-a513-d6d8cff16d86 None admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:19.505340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1363-f1b5-4ffa-a513-d6d8cff16d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:19.505610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc1363-f1b5-4ffa-a513-d6d8cff16d86 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/328ec1aac43349b28a72d5df19fd2cdd/roles/4981f801f6b14138984b3ae833a5797a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:19.520018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc1363-f1b5-4ffa-a513-d6d8cff16d86 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=328ec1aac43349b28a72d5df19fd2cdd, role_id=4981f801f6b14138984b3ae833a5797a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:19.521482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc1363-f1b5-4ffa-a513-d6d8cff16d86 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:19.597246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 609/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:58:19 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/328ec1aac43349b28a72d5df19fd2cdd/roles/4981f801f6b14138984b3ae833a5797a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:58:21.594383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e856446-b14a-4edc-8b3b-c130de928dc7 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.594933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e856446-b14a-4edc-8b3b-c130de928dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.594933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e856446-b14a-4edc-8b3b-c130de928dc7 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.596238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 624/1233] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:21.601344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237501b-863a-45d0-b792-0d9c8c56d67c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.602053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237501b-863a-45d0-b792-0d9c8c56d67c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.602053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237501b-863a-45d0-b792-0d9c8c56d67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.622004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6237501b-863a-45d0-b792-0d9c8c56d67c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:21.623213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6237501b-863a-45d0-b792-0d9c8c56d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:21.689364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 610/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:58:21.712057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e856446-b14a-4edc-8b3b-c130de928dc7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:21.737931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:21.818271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e856446-b14a-4edc-8b3b-c130de928dc7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:21.851265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:21.851915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.852200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.852517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.862975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:21.868005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:21.869249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:21.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 625/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:21.875937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237501b-863a-45d0-b792-0d9c8c56d67c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:21.921922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:21.973897 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237501b-863a-45d0-b792-0d9c8c56d67c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.007846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.008488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.008770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.009311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.010059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] RBAC: Adding query filter params (name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:22.010364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.011724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.019977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 611/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:22.025813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:22.109965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:22.125886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920c8e3-20af-407c-9f7c-a55276ee9f0b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.162867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.163505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.163787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.164060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.174480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.175926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:22.191010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:22.192038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:22.193139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:22.193139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:22.193139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:22.193139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:22.193139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 626/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:22.198116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:22.297135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c547-22f9-42ff-a0ef-e91c0417e950 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.306514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:22.331669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.333652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.333652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.333652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.333652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:22.334122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.335183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.347659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 612/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:22.353789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:22.453179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631995c-d3c4-4843-9e82-acb15cb85c39 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.486437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.487074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.487357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.487632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.490418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:22.495487 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.497001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:22.509311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:22.510387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:22.511610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 627/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:22.516248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:22.612797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07272339-01ef-4307-82d9-5f7456db1e7f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.648481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.649099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.649381 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.649657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.650452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:22.650760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.652255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.669123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 613/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:22.675354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:22.774097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05b6fe-6743-49fc-a3dd-ad7eb9919185 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.808010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.808658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.808935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.809201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.870176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:22.876684 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.878151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.973953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 628/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:22.979801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:23.006608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:23.077388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ae5eb-74bb-4a46-a282-dda8a0abcbb3 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:23.111182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:23.113866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:23.113866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:23.113866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:23.122614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:23.124047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:23.138968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:23.140621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:23.142019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:23.142019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:23.142019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:23.142019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:23.142019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 614/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:23.146527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:23.238213 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:23.244085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fcbad-88a9-4bd4-9d12-9bd57c851553 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:23.277473 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:23.278079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:23.278364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:23.278640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:23.279335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:23.280242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:23.281761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:23.292754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 629/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:23.298443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:23.378121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:23.400167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f21a7-dc20-42c0-a8d5-7a1872e26d98 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:23.433076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:23.433702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:23.433983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:23.434341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:23.440963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:23.442412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:23.454179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:23.455188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:23.456429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 615/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:23.461434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:23.560703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81473c5c-215c-4b8e-b79f-a60b59a98da9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:23.594046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe6397-c47c-4037-93a9-3850fd56d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:23.594590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe6397-c47c-4037-93a9-3850fd56d6ee None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:23.594869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe6397-c47c-4037-93a9-3850fd56d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:23.595140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe6397-c47c-4037-93a9-3850fd56d6ee None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:23.595886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe6397-c47c-4037-93a9-3850fd56d6ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:23.596191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe6397-c47c-4037-93a9-3850fd56d6ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:23.597638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe6397-c47c-4037-93a9-3850fd56d6ee None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:23.609281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 630/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:23.615376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:23.713953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b81c-0c1d-474f-ac7c-23ee47a171b7 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:23.747255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e69933-8572-451a-99c5-341ea3b60c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:23.747860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e69933-8572-451a-99c5-341ea3b60c1a None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:23.748141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e69933-8572-451a-99c5-341ea3b60c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:23.748970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e69933-8572-451a-99c5-341ea3b60c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:23.749753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e69933-8572-451a-99c5-341ea3b60c1a None admin] RBAC: Adding query filter params (role.id=4981f801f6b14138984b3ae833a5797a, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=328ec1aac43349b28a72d5df19fd2cdd) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:23.750059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e69933-8572-451a-99c5-341ea3b60c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:23.751481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e69933-8572-451a-99c5-341ea3b60c1a None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:23.757113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 616/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:23 2022] GET /identity/v3/role_assignments?user.id=328ec1aac43349b28a72d5df19fd2cdd&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=4981f801f6b14138984b3ae833a5797a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:25.795072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c596f-0cb0-4012-bb19-012d698326d4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.795617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c596f-0cb0-4012-bb19-012d698326d4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.795617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c596f-0cb0-4012-bb19-012d698326d4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:25.797022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 631/1247] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:25.802876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae79ba-91bb-44b5-aee9-85729fd8260c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.803887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae79ba-91bb-44b5-aee9-85729fd8260c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.803887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae79ba-91bb-44b5-aee9-85729fd8260c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:25.809763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:25.822461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-deae79ba-91bb-44b5-aee9-85729fd8260c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:25.823648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-deae79ba-91bb-44b5-aee9-85729fd8260c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:25.888363 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 617/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:25.911353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c596f-0cb0-4012-bb19-012d698326d4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:25.929641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.011517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c596f-0cb0-4012-bb19-012d698326d4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.044026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.045357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.045620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.045890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.055806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:26.060768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:26.064273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:26.065185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 632/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:58:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:26.068098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae79ba-91bb-44b5-aee9-85729fd8260c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.165261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae79ba-91bb-44b5-aee9-85729fd8260c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.193264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.198282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.198907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.199189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.199454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.200184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:26.200522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.202409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.210299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 618/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:26.216001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.291771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.313500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79419236-d66d-4197-83af-31f3f9b2202f None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.347728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.348366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.348675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.348950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.358720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.360263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:26.374514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:26.375561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:26.376726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:26.376726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:26.376726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:26.376726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:26.376726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 633/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:26.381370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.480556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b501d-f35f-4bc6-9866-359d56f6687f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.518748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.519422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.519707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.520014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.520815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:26.521123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.522487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.534332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 619/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:26.540043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.638938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca81a8f-eaff-43d2-b48f-8401355bf60c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.664527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.672115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.673925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.673925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.673925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.680456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.681877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:26.695311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:26.696430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:26.697433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 634/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:26.703151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.758846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.803498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595522e-a302-4672-abc3-c3065a0ea25d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.839576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.840193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.840507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.840788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.841659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:26.841967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.843384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.861295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 620/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:26.867112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.969741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1aad5-01a3-44ab-8989-f1a6284b546c None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:27.001894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:27.002507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:27.002788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:27.003063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:27.057459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:27.069308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:27.070751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:27.167819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 635/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:27.173727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:27.273548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42f338-cfbc-4527-ac16-3555eebd064e None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:27.286141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:27.306146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:27.306775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:27.307052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:27.307320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:27.317975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:27.319405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:27.335137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:27.336294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:27.343878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:27.343878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:27.343878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:27.343878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:27.343878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 621/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:27.344832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:27.428880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:27.441371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228578-1696-4e99-a36f-34904beff84d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:27.475622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:27.476234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:27.476549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:27.476823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:27.477524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:27.477825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:27.479276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:27.491157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 636/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:27.496946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:27.596757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aa994-0c8c-4600-873e-9dd5b9fd9388 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:27.630302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:27.630932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:27.631209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:27.631481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:27.638187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:27.639677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:27.651030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:27.652051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:27.653171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 622/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:27.658102 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:27.758902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4cc61-1627-48e8-8e17-2d35598cd750 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:27.795456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1661f0-b267-4e78-802a-5c002b8bb845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:27.796067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1661f0-b267-4e78-802a-5c002b8bb845 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:27.796345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1661f0-b267-4e78-802a-5c002b8bb845 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:27.796670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1661f0-b267-4e78-802a-5c002b8bb845 None admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:27.797458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1661f0-b267-4e78-802a-5c002b8bb845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:27.797766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1661f0-b267-4e78-802a-5c002b8bb845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:27.799217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1661f0-b267-4e78-802a-5c002b8bb845 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:27.811679 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 637/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:27.817902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:27.842656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:27.912696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b457f2e-a3e7-4d28-97df-b2c6b1b56271 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:27.945944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8275c5-0e8e-43cd-b195-c97808fc9928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:27.946554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8275c5-0e8e-43cd-b195-c97808fc9928 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:27.946838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8275c5-0e8e-43cd-b195-c97808fc9928 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:27.947130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8275c5-0e8e-43cd-b195-c97808fc9928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:27.947933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8275c5-0e8e-43cd-b195-c97808fc9928 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=328ec1aac43349b28a72d5df19fd2cdd) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:27.948241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8275c5-0e8e-43cd-b195-c97808fc9928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:27.949693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8275c5-0e8e-43cd-b195-c97808fc9928 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:27.955582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 623/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:27 2022] GET /identity/v3/role_assignments?user.id=328ec1aac43349b28a72d5df19fd2cdd&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=7912a7addff34d9582eb2933d7110ebc => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:29.964145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca4d1-da3a-4489-934d-a9d279bac4d6 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:29.964729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca4d1-da3a-4489-934d-a9d279bac4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:29.964729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca4d1-da3a-4489-934d-a9d279bac4d6 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:29.966673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 638/1261] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:29.971910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6282e-5af7-4254-aa21-0f5bfc63ff3f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:29.972324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6282e-5af7-4254-aa21-0f5bfc63ff3f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:29.972481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6282e-5af7-4254-aa21-0f5bfc63ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:29.990220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bbb6282e-5af7-4254-aa21-0f5bfc63ff3f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:29.991385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bbb6282e-5af7-4254-aa21-0f5bfc63ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:30.057656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 624/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:30.080259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ca4d1-da3a-4489-934d-a9d279bac4d6 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.091653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.178453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ca4d1-da3a-4489-934d-a9d279bac4d6 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.211584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.212211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.212519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.216423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.223287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:30.228351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:30.229536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:30.230554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 639/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:30.235719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6282e-5af7-4254-aa21-0f5bfc63ff3f None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.254542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.332668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6282e-5af7-4254-aa21-0f5bfc63ff3f None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.365823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a64903-8877-455e-9953-896e74a09195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.366438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a64903-8877-455e-9953-896e74a09195 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.366720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a64903-8877-455e-9953-896e74a09195 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.366995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a64903-8877-455e-9953-896e74a09195 None admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.367704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a64903-8877-455e-9953-896e74a09195 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:30.368009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a64903-8877-455e-9953-896e74a09195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.369440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a64903-8877-455e-9953-896e74a09195 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.377825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 625/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:30.383917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.449243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.481115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2cf0-a7a4-4e1c-8082-3a7a1285277d None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.515511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.516174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.516492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.516773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.526707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.528209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:30.542820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:30.543759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:30.544723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:30.544723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:30.544723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:30.544723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:30.544723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 640/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:30.550049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a64903-8877-455e-9953-896e74a09195 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.631888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.650905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a64903-8877-455e-9953-896e74a09195 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.683447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.684952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.685234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.689411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.689411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:30.689411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.689411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.700206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 626/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:30.705977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.804242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721e05e-5b15-4897-a859-8255115193d3 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.824030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.837752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.838386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.838665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.838937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.846487 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.847945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:30.859715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:30.860869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:30.861827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 641/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:30.866977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.966300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2765b-9711-4946-8a27-a98c6eee0123 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.000427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.001222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.001597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.001875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.003072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:31.003606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.005535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.015801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:31.024372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 627/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:31.030363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.129836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd8e3f-b27d-4056-931f-dbcba855a912 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.162650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.163275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.163555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.163827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.173928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.175379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:31.193140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:31.194269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:31.195387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:31.195387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:31.195387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:31.195387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:31.195387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 642/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:31.200957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.297979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852987cc-6c40-4917-b68d-32bbfe33106f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.332450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.333087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.333372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.333652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.334399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:31.334713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.336084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.347945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 628/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:31.353810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.386148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:31.454286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed1a48-bd01-47a8-b748-6dc4d774693b None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.487549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.488175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.488481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.488758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.495253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.497348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:31.509101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:31.510121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:31.511211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 643/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:31.516221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.574650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:31.614403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d425b-0b47-45a1-9fc2-5f393987bc12 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.652481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d80ee-afae-4c7e-86d8-6342f6c07874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.652481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d80ee-afae-4c7e-86d8-6342f6c07874 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.652481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d80ee-afae-4c7e-86d8-6342f6c07874 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.652481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d80ee-afae-4c7e-86d8-6342f6c07874 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.652905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d80ee-afae-4c7e-86d8-6342f6c07874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:31.652905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d80ee-afae-4c7e-86d8-6342f6c07874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.652905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d80ee-afae-4c7e-86d8-6342f6c07874 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.662872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 629/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:31.669549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.760028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:31.775017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd16d6-4533-4fb6-88ca-98a458a142d9 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.808644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701317a-68fe-474b-a864-863077d16fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.809305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701317a-68fe-474b-a864-863077d16fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.809586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701317a-68fe-474b-a864-863077d16fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.809859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701317a-68fe-474b-a864-863077d16fc2 None admin] PATH_INFO: `/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/328ec1aac43349b28a72d5df19fd2cdd/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.811775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701317a-68fe-474b-a864-863077d16fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=177080d4a05f46f1b40d6f62a83f7b79, user_id=328ec1aac43349b28a72d5df19fd2cdd, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.813229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701317a-68fe-474b-a864-863077d16fc2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.965114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 644/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Jul 27 19:58:31 2022] PUT /identity/v3/projects/177080d4a05f46f1b40d6f62a83f7b79/users/328ec1aac43349b28a72d5df19fd2cdd/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:58:33.934406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430010ac-f75f-4b1a-ac68-db6faaa0823b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:33.934927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430010ac-f75f-4b1a-ac68-db6faaa0823b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:33.934927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430010ac-f75f-4b1a-ac68-db6faaa0823b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:33.936355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 630/1274] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:33.941499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653890-d7f5-419d-9276-495dbb29f9dc None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:33.941934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653890-d7f5-419d-9276-495dbb29f9dc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:33.942029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27653890-d7f5-419d-9276-495dbb29f9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:33.960397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-27653890-d7f5-419d-9276-495dbb29f9dc None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:33.961589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-27653890-d7f5-419d-9276-495dbb29f9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:34.026223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 645/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:34.050712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430010ac-f75f-4b1a-ac68-db6faaa0823b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.151507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430010ac-f75f-4b1a-ac68-db6faaa0823b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.168646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.188160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.188160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.188160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.188160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.198551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:34.204501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:34.205633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:34.206681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 631/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:34.211813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27653890-d7f5-419d-9276-495dbb29f9dc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.293080 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.310864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27653890-d7f5-419d-9276-495dbb29f9dc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.343618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102361fa-8afd-4406-9653-708097a29bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.344228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102361fa-8afd-4406-9653-708097a29bba None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.344545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102361fa-8afd-4406-9653-708097a29bba None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.344820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102361fa-8afd-4406-9653-708097a29bba None admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.345533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102361fa-8afd-4406-9653-708097a29bba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:34.345833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102361fa-8afd-4406-9653-708097a29bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.347191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102361fa-8afd-4406-9653-708097a29bba None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.355175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 646/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:34.361311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.459747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92558d80-15bf-423e-805e-1b729c7c273f None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.493026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.493656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.493937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.494211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.504641 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.506088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:34.520588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:34.521608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:34.522680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:34.522680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:34.522680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:34.522680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:34.522680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 632/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:34.527408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102361fa-8afd-4406-9653-708097a29bba None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.627460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102361fa-8afd-4406-9653-708097a29bba None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.660324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.660965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.661245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.661518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.662212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:34.662516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.663878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.675888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 647/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:34.681567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.711784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.778048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4df0-eafa-4efa-814e-f044dc03538b None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.811720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.812347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.812655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.812924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.825190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.826643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:34.837701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:34.842647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:34.843794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 633/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:58:34.845251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.849139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.942515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8679163-79bc-40cf-ad59-59820f3151c2 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.975577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.976125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.976429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.976709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.977509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:34.977810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.979228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.996229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 648/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:35.001960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.096845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.107413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960797df-c9cb-48ac-a023-32d74a8ba099 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.140319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.140969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.141252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.141530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.208634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.210392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.297954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.307613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 634/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:35.313613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.375032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.412269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962572e5-42a9-46a3-936a-2773b7dbde23 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.444870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.445539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.445840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.446112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.456275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.457783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:35.472630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:35.473669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:35.474593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:35.474593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:35.474593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:35.474593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:35.474593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 649/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:35.479746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.577087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a45be1-e9b2-48b4-b6ab-6ab7b9704df6 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.610459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.611065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.611343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.611616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.612297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:35.612625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.613981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.625465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 635/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:35.631251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.729080 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b44a7-dd0b-4bb1-80bf-3fd323a8ce4e None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.742818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.764003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.764670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.764951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.765224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.771729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.773199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:35.784359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:35.785510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 19:58:35.786666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 650/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:35.791380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.842123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.893666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921aa13-4a96-4613-a93f-3b1192874c2d None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.927131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fec74-89de-4553-bb14-1fb09d442029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.931035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fec74-89de-4553-bb14-1fb09d442029 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.931675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fec74-89de-4553-bb14-1fb09d442029 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.932061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fec74-89de-4553-bb14-1fb09d442029 None admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.932857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fec74-89de-4553-bb14-1fb09d442029 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:35.933777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fec74-89de-4553-bb14-1fb09d442029 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.935260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fec74-89de-4553-bb14-1fb09d442029 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.947184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 636/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:35.953568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:36.052439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8d7a7-86fd-44f7-a0a3-aca456b4de95 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:36.088323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329113c4-73c1-4242-ac38-427e547480e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:36.088990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329113c4-73c1-4242-ac38-427e547480e6 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:36.089303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329113c4-73c1-4242-ac38-427e547480e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:36.089579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329113c4-73c1-4242-ac38-427e547480e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:36.090370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329113c4-73c1-4242-ac38-427e547480e6 None admin] RBAC: Adding query filter params (role.id=7912a7addff34d9582eb2933d7110ebc, scope.project.id=177080d4a05f46f1b40d6f62a83f7b79, user.id=328ec1aac43349b28a72d5df19fd2cdd) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:36.090683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329113c4-73c1-4242-ac38-427e547480e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:36.092104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329113c4-73c1-4242-ac38-427e547480e6 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:36.097940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 651/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/role_assignments?user.id=328ec1aac43349b28a72d5df19fd2cdd&scope.project.id=177080d4a05f46f1b40d6f62a83f7b79&role.id=7912a7addff34d9582eb2933d7110ebc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:38.136970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afbbfe-938c-4f2e-9485-4191dcd676c2 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.137552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afbbfe-938c-4f2e-9485-4191dcd676c2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.137552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afbbfe-938c-4f2e-9485-4191dcd676c2 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.138865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 637/1288] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:38.144136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63770293-7a58-4270-9f40-0e458b5314dd None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.144552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63770293-7a58-4270-9f40-0e458b5314dd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.144729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63770293-7a58-4270-9f40-0e458b5314dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.162931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-63770293-7a58-4270-9f40-0e458b5314dd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:38.164116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-63770293-7a58-4270-9f40-0e458b5314dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:38.192387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:38.229063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 652/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:38.250352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afbbfe-938c-4f2e-9485-4191dcd676c2 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.352141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afbbfe-938c-4f2e-9485-4191dcd676c2 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.362960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:38.385330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.386481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.386764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.387087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.387665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.389058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:38.397729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:38.398992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:38.399863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 638/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:38.405550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63770293-7a58-4270-9f40-0e458b5314dd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.500659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63770293-7a58-4270-9f40-0e458b5314dd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.533537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf017a-75b6-4d10-ae36-c154c0ba2934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.534153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf017a-75b6-4d10-ae36-c154c0ba2934 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.534433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf017a-75b6-4d10-ae36-c154c0ba2934 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.534706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf017a-75b6-4d10-ae36-c154c0ba2934 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.535378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf017a-75b6-4d10-ae36-c154c0ba2934 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:38.535683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf017a-75b6-4d10-ae36-c154c0ba2934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.537079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf017a-75b6-4d10-ae36-c154c0ba2934 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.541754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 653/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:38.547552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.646341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb4f38-9183-4dc2-9b81-729a00cd16e2 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.678529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809713e-14ce-46e0-b2b1-2e9478340175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.679148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809713e-14ce-46e0-b2b1-2e9478340175 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.679417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809713e-14ce-46e0-b2b1-2e9478340175 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.679688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809713e-14ce-46e0-b2b1-2e9478340175 None admin] PATH_INFO: `/v3/services` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.680358 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809713e-14ce-46e0-b2b1-2e9478340175 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:38.680692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809713e-14ce-46e0-b2b1-2e9478340175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.682045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809713e-14ce-46e0-b2b1-2e9478340175 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.687024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 639/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:40.670165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cc035-910e-46f6-8193-6693f343989a None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:40.670717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cc035-910e-46f6-8193-6693f343989a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:40.670717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cc035-910e-46f6-8193-6693f343989a None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:40.671984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 654/1293] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:40.677188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb069e-3f10-4bd9-848d-a220247cabc3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:40.677602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb069e-3f10-4bd9-848d-a220247cabc3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:40.677758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb069e-3f10-4bd9-848d-a220247cabc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:40.695403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4ceb069e-3f10-4bd9-848d-a220247cabc3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:40.696624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4ceb069e-3f10-4bd9-848d-a220247cabc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:40.725921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:40.762299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 640/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:40.784508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cc035-910e-46f6-8193-6693f343989a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:40.817885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:40.887655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cc035-910e-46f6-8193-6693f343989a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:40.921881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ce28f-817b-44af-9116-a34d8a331aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:40.922421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce28f-817b-44af-9116-a34d8a331aab None admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:40.922698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce28f-817b-44af-9116-a34d8a331aab None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:40.922968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ce28f-817b-44af-9116-a34d8a331aab None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:40.923660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ce28f-817b-44af-9116-a34d8a331aab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:40.925042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ce28f-817b-44af-9116-a34d8a331aab None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:41.014092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 655/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Jul 27 19:58:40 2022] POST /identity/v3/services => generated 246 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:58:42.989094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bef4-47f7-4a05-8eba-7ca43d1a4443 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:42.989626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bef4-47f7-4a05-8eba-7ca43d1a4443 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:42.989626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bef4-47f7-4a05-8eba-7ca43d1a4443 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:42.990975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 641/1296] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:42.996509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5e9cd-5d40-465b-9a9f-147821264478 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:42.996904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5e9cd-5d40-465b-9a9f-147821264478 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:42.997077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5e9cd-5d40-465b-9a9f-147821264478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:43.015385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-44e5e9cd-5d40-465b-9a9f-147821264478 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:43.018429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-44e5e9cd-5d40-465b-9a9f-147821264478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:43.085969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 656/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:43.107627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bef4-47f7-4a05-8eba-7ca43d1a4443 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:43.210358 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bef4-47f7-4a05-8eba-7ca43d1a4443 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:43.244341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:43.244947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:43.245229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:43.245506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:43.246139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:43.247595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:43.258474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:43.265080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:43.266166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 642/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:58:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:43.267184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5e9cd-5d40-465b-9a9f-147821264478 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:43.284628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:43.362369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5e9cd-5d40-465b-9a9f-147821264478 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:43.395696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:43.396307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:43.396618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:43.396895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:43.397578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:43.397882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:43.399241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:43.404960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 657/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:43.410914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:43.442601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:43.508035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a640-7a56-466c-968a-cda69240fa07 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:43.540767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8674-287c-45d2-a0b2-95198714d5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:43.541367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8674-287c-45d2-a0b2-95198714d5db None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:43.541643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8674-287c-45d2-a0b2-95198714d5db None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:43.541912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8674-287c-45d2-a0b2-95198714d5db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:43.542651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8674-287c-45d2-a0b2-95198714d5db None admin] RBAC: Adding query filter params (interface=public, service_id=0b342fdd14884b8b90c128d94c68fa14, region_id=RegionOne) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:43.542955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8674-287c-45d2-a0b2-95198714d5db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:43.544301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8674-287c-45d2-a0b2-95198714d5db None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:43.550881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 643/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Jul 27 19:58:43 2022] GET /identity/v3/endpoints?service_id=0b342fdd14884b8b90c128d94c68fa14&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:43.556727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:43.639677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:43.657596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946ff21-0671-4dcc-af5e-83337d993304 None admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:43.689510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f7e5c-723a-494b-aad3-55f47ce784c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:43.690118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7e5c-723a-494b-aad3-55f47ce784c4 None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:43.690399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7e5c-723a-494b-aad3-55f47ce784c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:43.690672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7e5c-723a-494b-aad3-55f47ce784c4 None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:43.691325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f7e5c-723a-494b-aad3-55f47ce784c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:43.692715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f7e5c-723a-494b-aad3-55f47ce784c4 None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:43.699993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 658/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Jul 27 19:58:43 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 19:58:45.680453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208028b-3dec-40b9-b662-e744317d04b3 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:45.680986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208028b-3dec-40b9-b662-e744317d04b3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:45.680986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208028b-3dec-40b9-b662-e744317d04b3 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:45.682325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 644/1302] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 19:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:45.687476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67f559-d214-4ea5-9a2d-38bcacfb5484 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:45.687845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67f559-d214-4ea5-9a2d-38bcacfb5484 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:45.688022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67f559-d214-4ea5-9a2d-38bcacfb5484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:45.706241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ed67f559-d214-4ea5-9a2d-38bcacfb5484 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:45.707427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ed67f559-d214-4ea5-9a2d-38bcacfb5484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:45.773264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 659/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:45.795626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208028b-3dec-40b9-b662-e744317d04b3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:45.889071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:45.895506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208028b-3dec-40b9-b662-e744317d04b3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:45.927676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:45.928295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:45.928617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:45.928884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:45.929497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:45.930915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:45.939772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:45.940924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 19:58:45.942158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 645/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Jul 27 19:58:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:45.947396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67f559-d214-4ea5-9a2d-38bcacfb5484 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:46.023391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:46.044591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67f559-d214-4ea5-9a2d-38bcacfb5484 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:46.078125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13079437-baa9-488c-be95-c7f2f76bd18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:46.078736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13079437-baa9-488c-be95-c7f2f76bd18e None admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:46.079018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13079437-baa9-488c-be95-c7f2f76bd18e None admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:46.079320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13079437-baa9-488c-be95-c7f2f76bd18e None admin] PATH_INFO: `/v3/services` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:46.080021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13079437-baa9-488c-be95-c7f2f76bd18e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:46.080324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13079437-baa9-488c-be95-c7f2f76bd18e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:46.081738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13079437-baa9-488c-be95-c7f2f76bd18e None admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:46.087100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 660/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:46.092615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:46.190723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b038f-3dda-4a56-9168-eb882dfbadd1 None admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:46.223504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b2648-5d96-44f2-a2f5-7f14cef03c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:46.224131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b2648-5d96-44f2-a2f5-7f14cef03c7f None admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:46.224434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b2648-5d96-44f2-a2f5-7f14cef03c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:46.224710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b2648-5d96-44f2-a2f5-7f14cef03c7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:46.225407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b2648-5d96-44f2-a2f5-7f14cef03c7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:46.226753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b2648-5d96-44f2-a2f5-7f14cef03c7f None admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:46.327441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 646/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 19:58:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:58:56.088107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=30 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:56.322325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:46.985307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291f99c-a4c3-4908-9b45-3faeb5159a42 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:46.985908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291f99c-a4c3-4908-9b45-3faeb5159a42 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:46.985908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291f99c-a4c3-4908-9b45-3faeb5159a42 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:46.988824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 661/1307] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:04:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:46.992406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c40852-bb79-4a12-bc8f-6d8903caa4a7 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:46.992942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c40852-bb79-4a12-bc8f-6d8903caa4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:46.993038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c40852-bb79-4a12-bc8f-6d8903caa4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:47.112000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-06c40852-bb79-4a12-bc8f-6d8903caa4a7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:47.113477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-06c40852-bb79-4a12-bc8f-6d8903caa4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:47.304420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 647/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:47.349270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4561f-37c1-4bd8-8036-3b213bbe9bfa None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:47.350010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4561f-37c1-4bd8-8036-3b213bbe9bfa None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:47.350522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4561f-37c1-4bd8-8036-3b213bbe9bfa None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:47.352141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 662/1309] 10.222.0.47 () {54 vars in 965 bytes} [Wed Jul 27 20:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:47.359404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08eb7-a857-46b3-a69a-8c58816476bc None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:47.359877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08eb7-a857-46b3-a69a-8c58816476bc None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:47.361961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08eb7-a857-46b3-a69a-8c58816476bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:47.443403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-88b08eb7-a857-46b3-a69a-8c58816476bc None None] MFA Rules not processed for user `54aedfbb64c54c589003b335fde2a17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:47.444815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-88b08eb7-a857-46b3-a69a-8c58816476bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:47.466918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:47.525609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 648/1310] 10.222.0.47 () {58 vars in 1062 bytes} [Wed Jul 27 20:04:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:04:47.534465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4561f-37c1-4bd8-8036-3b213bbe9bfa None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:47.536208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0ac4561f-37c1-4bd8-8036-3b213bbe9bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:47.650737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4561f-37c1-4bd8-8036-3b213bbe9bfa None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:47.727166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfca610-20ad-4ebe-acda-0ee345386406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:47.727904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfca610-20ad-4ebe-acda-0ee345386406 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:47.728190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfca610-20ad-4ebe-acda-0ee345386406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:47.728496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfca610-20ad-4ebe-acda-0ee345386406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:47.730140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8cfca610-20ad-4ebe-acda-0ee345386406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:47.784415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:47.850195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfca610-20ad-4ebe-acda-0ee345386406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:47.852368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfca610-20ad-4ebe-acda-0ee345386406 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:47.875646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 663/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:04:50.172201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad881-66cf-454c-ac77-fb051ee6c689 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:50.172800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad881-66cf-454c-ac77-fb051ee6c689 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:50.172800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ad881-66cf-454c-ac77-fb051ee6c689 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:50.174186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 649/1312] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:50.179862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadefc0b-b311-4f58-b2fb-335d0748a719 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:50.180920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadefc0b-b311-4f58-b2fb-335d0748a719 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:50.180920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadefc0b-b311-4f58-b2fb-335d0748a719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:50.258244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-dadefc0b-b311-4f58-b2fb-335d0748a719 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:50.260067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dadefc0b-b311-4f58-b2fb-335d0748a719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:50.361089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 664/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:50.414401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e5033-4d8e-464c-88de-b4a792902407 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:50.414707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e5033-4d8e-464c-88de-b4a792902407 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:50.414989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e5033-4d8e-464c-88de-b4a792902407 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:50.416337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 650/1314] 10.222.0.47 () {54 vars in 965 bytes} [Wed Jul 27 20:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:50.422591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa0a22-4d8f-4bdf-a1e9-086116443f96 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:50.422802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa0a22-4d8f-4bdf-a1e9-086116443f96 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:50.423112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa0a22-4d8f-4bdf-a1e9-086116443f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:50.436271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:50.501016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-77aa0a22-4d8f-4bdf-a1e9-086116443f96 None None] MFA Rules not processed for user `54aedfbb64c54c589003b335fde2a17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:50.502240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-77aa0a22-4d8f-4bdf-a1e9-086116443f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:50.582458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 665/1315] 10.222.0.47 () {58 vars in 1062 bytes} [Wed Jul 27 20:04:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:04:50.592000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e5033-4d8e-464c-88de-b4a792902407 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:50.593601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2b7e5033-4d8e-464c-88de-b4a792902407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:50.693319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e5033-4d8e-464c-88de-b4a792902407 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:50.764623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:50.767081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974eda7f-914c-4deb-a9ed-6adbb04402ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:50.767772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974eda7f-914c-4deb-a9ed-6adbb04402ab service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:50.768049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974eda7f-914c-4deb-a9ed-6adbb04402ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:50.768320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974eda7f-914c-4deb-a9ed-6adbb04402ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:50.769955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-974eda7f-914c-4deb-a9ed-6adbb04402ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:50.875930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974eda7f-914c-4deb-a9ed-6adbb04402ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:50.877366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974eda7f-914c-4deb-a9ed-6adbb04402ab service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:50.895459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 651/1316] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:04:53.601500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8997c-c483-4f0f-becf-7e41a8489419 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:53.602014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8997c-c483-4f0f-becf-7e41a8489419 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:53.602114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8997c-c483-4f0f-becf-7e41a8489419 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:53.603476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 666/1317] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:53.608660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584873a-d29d-40e9-813b-7263008e36be None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:53.609045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584873a-d29d-40e9-813b-7263008e36be None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:53.609479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584873a-d29d-40e9-813b-7263008e36be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:53.674399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3584873a-d29d-40e9-813b-7263008e36be None None] MFA Rules not processed for user `a3dfb349996e4ef189c4b08a2a09941e`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:53.675621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3584873a-d29d-40e9-813b-7263008e36be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:53.726625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:53.770661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 652/1318] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:04:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:53.810211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8997c-c483-4f0f-becf-7e41a8489419 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:53.811795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b8c8997c-c483-4f0f-becf-7e41a8489419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:53.917466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8997c-c483-4f0f-becf-7e41a8489419 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:53.920259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:53.991259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5ab53-0e0b-41a0-9f2f-18f426da57af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:53.991873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ab53-0e0b-41a0-9f2f-18f426da57af service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:53.992154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ab53-0e0b-41a0-9f2f-18f426da57af service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:53.992454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ab53-0e0b-41a0-9f2f-18f426da57af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:53.994595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-eba5ab53-0e0b-41a0-9f2f-18f426da57af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:54.099584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5ab53-0e0b-41a0-9f2f-18f426da57af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:54.101034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5ab53-0e0b-41a0-9f2f-18f426da57af service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:54.119328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 667/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:04:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:04:56.924518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd356300-6a73-49b6-b1eb-ad790ad9d34e None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:56.925075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd356300-6a73-49b6-b1eb-ad790ad9d34e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:56.925075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd356300-6a73-49b6-b1eb-ad790ad9d34e None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:56.926404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 653/1320] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:56.932048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a594-675f-423b-aa64-30c3b66b412f None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:56.932495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a594-675f-423b-aa64-30c3b66b412f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:56.932629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a594-675f-423b-aa64-30c3b66b412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:56.972897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:56.997732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-8f35a594-675f-423b-aa64-30c3b66b412f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:56.998932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8f35a594-675f-423b-aa64-30c3b66b412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:57.093293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 668/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:04:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:57.131760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd356300-6a73-49b6-b1eb-ad790ad9d34e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:57.133344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-dd356300-6a73-49b6-b1eb-ad790ad9d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:57.235017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd356300-6a73-49b6-b1eb-ad790ad9d34e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:57.307278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fd4f-36e7-4e1e-af3c-a7b4f2f1e346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:57.307886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fd4f-36e7-4e1e-af3c-a7b4f2f1e346 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:57.308165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fd4f-36e7-4e1e-af3c-a7b4f2f1e346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:57.308470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fd4f-36e7-4e1e-af3c-a7b4f2f1e346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:57.310074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-11d3fd4f-36e7-4e1e-af3c-a7b4f2f1e346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:57.317367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:57.417894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3fd4f-36e7-4e1e-af3c-a7b4f2f1e346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:57.419313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3fd4f-36e7-4e1e-af3c-a7b4f2f1e346 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:57.437255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 654/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:04:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:00.466141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ce57-6405-4c47-8a8a-b9f65cb48134 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:00.466718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ce57-6405-4c47-8a8a-b9f65cb48134 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:00.466718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085ce57-6405-4c47-8a8a-b9f65cb48134 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:00.468049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 669/1323] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:00.473190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d6a6-1917-41aa-9a5f-cf5f759301e9 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:00.473629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d6a6-1917-41aa-9a5f-cf5f759301e9 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:00.473890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d6a6-1917-41aa-9a5f-cf5f759301e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:00.543801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6e60d6a6-1917-41aa-9a5f-cf5f759301e9 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:00.549244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6e60d6a6-1917-41aa-9a5f-cf5f759301e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:00.605183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:00.645446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 655/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:05:00.684163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ce57-6405-4c47-8a8a-b9f65cb48134 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:00.689133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c085ce57-6405-4c47-8a8a-b9f65cb48134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:00.795685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085ce57-6405-4c47-8a8a-b9f65cb48134 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:00.808599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:00.874194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c157eb-c11b-42b7-9cc0-e3b92dac5964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:00.874726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c157eb-c11b-42b7-9cc0-e3b92dac5964 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:00.875005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c157eb-c11b-42b7-9cc0-e3b92dac5964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:00.875275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c157eb-c11b-42b7-9cc0-e3b92dac5964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:00.876928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e0c157eb-c11b-42b7-9cc0-e3b92dac5964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:00.987493 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c157eb-c11b-42b7-9cc0-e3b92dac5964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:00.988976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c157eb-c11b-42b7-9cc0-e3b92dac5964 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:01.006700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 670/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:03.060263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3e54-293f-4d30-940f-632f8a5afda6 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:03.060843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3e54-293f-4d30-940f-632f8a5afda6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:03.060843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b3e54-293f-4d30-940f-632f8a5afda6 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:03.062185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 656/1326] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:03.067244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d4629-2c89-48b6-9293-da9c2b68bbe4 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:03.067670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d4629-2c89-48b6-9293-da9c2b68bbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:03.067797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d4629-2c89-48b6-9293-da9c2b68bbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:03.115343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:03.133380 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c77d4629-2c89-48b6-9293-da9c2b68bbe4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:03.135137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c77d4629-2c89-48b6-9293-da9c2b68bbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:03.230652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 671/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:03.267345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3e54-293f-4d30-940f-632f8a5afda6 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:03.268931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d25b3e54-293f-4d30-940f-632f8a5afda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:03.372623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b3e54-293f-4d30-940f-632f8a5afda6 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:03.446133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365a699-4e20-48de-8cf0-e8901badb7fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:03.446685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365a699-4e20-48de-8cf0-e8901badb7fc service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:03.446963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365a699-4e20-48de-8cf0-e8901badb7fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:03.447232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365a699-4e20-48de-8cf0-e8901badb7fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:03.448863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d365a699-4e20-48de-8cf0-e8901badb7fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:03.465815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:03.563495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365a699-4e20-48de-8cf0-e8901badb7fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:03.565015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365a699-4e20-48de-8cf0-e8901badb7fc service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:03.584008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 657/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:05.637163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bacfcb-ed96-478b-b9b6-155d732fec3d None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:05.637704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bacfcb-ed96-478b-b9b6-155d732fec3d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:05.637704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bacfcb-ed96-478b-b9b6-155d732fec3d None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:05.639053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 672/1329] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:05.644195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52947e95-01f7-4f6f-b172-5a9027e1b013 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:05.644626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52947e95-01f7-4f6f-b172-5a9027e1b013 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:05.644939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52947e95-01f7-4f6f-b172-5a9027e1b013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:05.718165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-52947e95-01f7-4f6f-b172-5a9027e1b013 None None] MFA Rules not processed for user `a3dfb349996e4ef189c4b08a2a09941e`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:05.719391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-52947e95-01f7-4f6f-b172-5a9027e1b013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:05.784867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:05.816170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 658/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:05.853746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bacfcb-ed96-478b-b9b6-155d732fec3d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:05.855438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-94bacfcb-ed96-478b-b9b6-155d732fec3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:05.962891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bacfcb-ed96-478b-b9b6-155d732fec3d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:05.984527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:06.038404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ee05f-4b78-4894-926d-5befd2a7b050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:06.039021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee05f-4b78-4894-926d-5befd2a7b050 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:06.039302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee05f-4b78-4894-926d-5befd2a7b050 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:06.039577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee05f-4b78-4894-926d-5befd2a7b050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:06.041266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bb6ee05f-4b78-4894-926d-5befd2a7b050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:06.151686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ee05f-4b78-4894-926d-5befd2a7b050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:06.153064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6ee05f-4b78-4894-926d-5befd2a7b050 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:06.171898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 673/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:08.818589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05780788-b176-451e-a6d6-5fd39d0bf581 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:08.819111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05780788-b176-451e-a6d6-5fd39d0bf581 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:08.819111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05780788-b176-451e-a6d6-5fd39d0bf581 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:08.820417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 659/1332] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:08.826032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c267f1-9044-41c7-92c1-ebe6a98d8ec9 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:08.826439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c267f1-9044-41c7-92c1-ebe6a98d8ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:08.826586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c267f1-9044-41c7-92c1-ebe6a98d8ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:08.894564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:08.897126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-69c267f1-9044-41c7-92c1-ebe6a98d8ec9 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:08.898353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-69c267f1-9044-41c7-92c1-ebe6a98d8ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:08.994810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 674/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:09.031245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05780788-b176-451e-a6d6-5fd39d0bf581 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:09.032816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-05780788-b176-451e-a6d6-5fd39d0bf581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:09.134654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05780788-b176-451e-a6d6-5fd39d0bf581 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:09.206455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9317604-1170-4ed6-a829-fa583255d5af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:09.207071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9317604-1170-4ed6-a829-fa583255d5af service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:09.207352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9317604-1170-4ed6-a829-fa583255d5af service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:09.207624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9317604-1170-4ed6-a829-fa583255d5af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:09.209273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a9317604-1170-4ed6-a829-fa583255d5af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:09.235953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:09.318169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9317604-1170-4ed6-a829-fa583255d5af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:09.319598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9317604-1170-4ed6-a829-fa583255d5af service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:09.337945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 660/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:11.371333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92fff-153b-4c42-9a16-4183db81e123 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:11.371837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92fff-153b-4c42-9a16-4183db81e123 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:11.371943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92fff-153b-4c42-9a16-4183db81e123 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:11.373288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 675/1335] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:11.378424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654fcac-ed53-4cac-bcff-8fa709637781 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:11.378893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654fcac-ed53-4cac-bcff-8fa709637781 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:11.378999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654fcac-ed53-4cac-bcff-8fa709637781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:11.443074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c654fcac-ed53-4cac-bcff-8fa709637781 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:11.444269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c654fcac-ed53-4cac-bcff-8fa709637781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:11.518186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:11.541425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 661/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:11.582623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92fff-153b-4c42-9a16-4183db81e123 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:11.584248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-72f92fff-153b-4c42-9a16-4183db81e123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:11.687932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92fff-153b-4c42-9a16-4183db81e123 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:11.716828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:11.762136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229b046-9d1c-437a-9f09-637fa5e3cd14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:11.762749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229b046-9d1c-437a-9f09-637fa5e3cd14 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:11.763032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229b046-9d1c-437a-9f09-637fa5e3cd14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:11.763308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229b046-9d1c-437a-9f09-637fa5e3cd14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:11.764971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d229b046-9d1c-437a-9f09-637fa5e3cd14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:11.873428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229b046-9d1c-437a-9f09-637fa5e3cd14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:11.874862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229b046-9d1c-437a-9f09-637fa5e3cd14 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:11.894810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 676/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:14.496929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a63bd-109b-4af6-82d3-1eee64a8ec31 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:14.497456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a63bd-109b-4af6-82d3-1eee64a8ec31 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:14.497571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a63bd-109b-4af6-82d3-1eee64a8ec31 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:14.498828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 662/1338] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:14.503943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1151c3-ef44-4ba6-89e8-fd42b835a37e None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:14.504411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1151c3-ef44-4ba6-89e8-fd42b835a37e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:14.504596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1151c3-ef44-4ba6-89e8-fd42b835a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:14.570075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-db1151c3-ef44-4ba6-89e8-fd42b835a37e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:14.571815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-db1151c3-ef44-4ba6-89e8-fd42b835a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:14.580076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:14.673383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 677/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:14.709560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a63bd-109b-4af6-82d3-1eee64a8ec31 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:14.711130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-911a63bd-109b-4af6-82d3-1eee64a8ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:14.824347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a63bd-109b-4af6-82d3-1eee64a8ec31 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:14.901528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8a17-c287-45d1-931b-5911594ce85c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:14.902152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8a17-c287-45d1-931b-5911594ce85c service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:14.902435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8a17-c287-45d1-931b-5911594ce85c service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:14.902711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8a17-c287-45d1-931b-5911594ce85c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:14.904358 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e86a8a17-c287-45d1-931b-5911594ce85c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:14.940652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:15.017476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8a17-c287-45d1-931b-5911594ce85c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:15.018887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8a17-c287-45d1-931b-5911594ce85c service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:15.039801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 663/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:19.299504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0ca56-3dc4-44cd-ab34-17d18ae52a90 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:19.299982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0ca56-3dc4-44cd-ab34-17d18ae52a90 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:19.300092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0ca56-3dc4-44cd-ab34-17d18ae52a90 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:19.301466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 678/1341] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:19.306890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac66c-c27a-4da1-9ce6-c926d8bc7584 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:19.307410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac66c-c27a-4da1-9ce6-c926d8bc7584 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:19.307518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac66c-c27a-4da1-9ce6-c926d8bc7584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:19.386729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-694ac66c-c27a-4da1-9ce6-c926d8bc7584 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:19.387941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-694ac66c-c27a-4da1-9ce6-c926d8bc7584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:19.479821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:19.491468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 664/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:19.530320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0ca56-3dc4-44cd-ab34-17d18ae52a90 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:19.531885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-35b0ca56-3dc4-44cd-ab34-17d18ae52a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:19.634861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0ca56-3dc4-44cd-ab34-17d18ae52a90 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:19.672309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:19.707660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15735ea6-bc43-4ffc-b09e-0ab694281f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:19.708273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15735ea6-bc43-4ffc-b09e-0ab694281f69 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:19.708582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15735ea6-bc43-4ffc-b09e-0ab694281f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:19.708867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15735ea6-bc43-4ffc-b09e-0ab694281f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:19.711207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-15735ea6-bc43-4ffc-b09e-0ab694281f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:19.820552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15735ea6-bc43-4ffc-b09e-0ab694281f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:19.822503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15735ea6-bc43-4ffc-b09e-0ab694281f69 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:19.841139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 679/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:22.818528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a378ee-c05f-4bae-887f-6a3ec54a4535 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:22.819055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a378ee-c05f-4bae-887f-6a3ec54a4535 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:22.819174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a378ee-c05f-4bae-887f-6a3ec54a4535 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:22.820498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 665/1344] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:22.826291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c0781-3189-4c6a-ab63-8fc0e8e19e17 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:22.826733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c0781-3189-4c6a-ab63-8fc0e8e19e17 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:22.826828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c0781-3189-4c6a-ab63-8fc0e8e19e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:22.895053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-eb6c0781-3189-4c6a-ab63-8fc0e8e19e17 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:22.896268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-eb6c0781-3189-4c6a-ab63-8fc0e8e19e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:22.914253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:22.994566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 680/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:05:23.031026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a378ee-c05f-4bae-887f-6a3ec54a4535 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:23.032636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b1a378ee-c05f-4bae-887f-6a3ec54a4535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:23.138258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a378ee-c05f-4bae-887f-6a3ec54a4535 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:23.211785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c6346-8604-4ea2-9029-34318ffd4dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:23.212459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c6346-8604-4ea2-9029-34318ffd4dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:23.212746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c6346-8604-4ea2-9029-34318ffd4dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:23.213021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c6346-8604-4ea2-9029-34318ffd4dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:23.214644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d27c6346-8604-4ea2-9029-34318ffd4dcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:23.258124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:23.326047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c6346-8604-4ea2-9029-34318ffd4dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:23.327479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c6346-8604-4ea2-9029-34318ffd4dcc service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:23.347915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 666/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:28.023324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc94713-ad49-4f0e-b5ab-ca43847c9ddd None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:28.023858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc94713-ad49-4f0e-b5ab-ca43847c9ddd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:28.023858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc94713-ad49-4f0e-b5ab-ca43847c9ddd None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:28.025836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 681/1347] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:28.031204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba8aca-3d6e-46fa-b9cf-6ebdbb30ddd6 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:28.031641 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba8aca-3d6e-46fa-b9cf-6ebdbb30ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:28.031724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba8aca-3d6e-46fa-b9cf-6ebdbb30ddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:28.102645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-36ba8aca-3d6e-46fa-b9cf-6ebdbb30ddd6 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:28.103520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-36ba8aca-3d6e-46fa-b9cf-6ebdbb30ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:28.200084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:28.202761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 667/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:28.241742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc94713-ad49-4f0e-b5ab-ca43847c9ddd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:28.243327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2bc94713-ad49-4f0e-b5ab-ca43847c9ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:28.348981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc94713-ad49-4f0e-b5ab-ca43847c9ddd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:28.396124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:28.421398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f41027-496d-4853-9714-14794153ce40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:28.422025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41027-496d-4853-9714-14794153ce40 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:28.422306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41027-496d-4853-9714-14794153ce40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:28.422580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41027-496d-4853-9714-14794153ce40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:28.424490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-03f41027-496d-4853-9714-14794153ce40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:28.532409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f41027-496d-4853-9714-14794153ce40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:28.533840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f41027-496d-4853-9714-14794153ce40 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:28.551921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 682/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:33.469483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f787-b1ea-4942-aa5f-892a87394393 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:33.469995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f787-b1ea-4942-aa5f-892a87394393 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:33.470094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730f787-b1ea-4942-aa5f-892a87394393 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:33.471639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 668/1350] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:33.476675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197b3db-895c-4358-8d12-dff315d0bce6 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:33.477106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197b3db-895c-4358-8d12-dff315d0bce6 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:33.477444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197b3db-895c-4358-8d12-dff315d0bce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:33.583393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3197b3db-895c-4358-8d12-dff315d0bce6 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:33.584640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3197b3db-895c-4358-8d12-dff315d0bce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:33.636209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:33.773201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 683/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:05:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:38.113374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f7ca-71e5-4e75-857a-eaf47bc0767e None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:38.113916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f7ca-71e5-4e75-857a-eaf47bc0767e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:38.114035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17f7ca-71e5-4e75-857a-eaf47bc0767e None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:38.115345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 669/1352] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:38.121431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a7c73-9465-4e30-bbcd-886aaa4524f8 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:38.121908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a7c73-9465-4e30-bbcd-886aaa4524f8 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:38.122192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a7c73-9465-4e30-bbcd-886aaa4524f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:38.205943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-511a7c73-9465-4e30-bbcd-886aaa4524f8 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:38.206692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-511a7c73-9465-4e30-bbcd-886aaa4524f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:38.283250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:38.322850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 684/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:05:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:39.401959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc10e27-fa78-43db-87ae-a66de522d98c None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:39.406432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc10e27-fa78-43db-87ae-a66de522d98c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:39.406432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc10e27-fa78-43db-87ae-a66de522d98c None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:39.406432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 670/1354] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:39.408989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b496-2353-40f3-8b92-1f70b523ff0c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:39.409791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b496-2353-40f3-8b92-1f70b523ff0c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:39.410281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904b496-2353-40f3-8b92-1f70b523ff0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:39.488458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6904b496-2353-40f3-8b92-1f70b523ff0c None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:39.489680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6904b496-2353-40f3-8b92-1f70b523ff0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:39.597404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 685/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:05:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:42.975570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50201128-eda3-4612-8e69-74327bb2a9cf None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:42.976575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50201128-eda3-4612-8e69-74327bb2a9cf None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:42.977107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50201128-eda3-4612-8e69-74327bb2a9cf None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:42.978695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 671/1356] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:42.984415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58223446-fae9-48e9-bd06-a2bbc84be6de None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:42.985467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58223446-fae9-48e9-bd06-a2bbc84be6de None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:42.986714 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58223446-fae9-48e9-bd06-a2bbc84be6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:43.034360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:43.070208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-58223446-fae9-48e9-bd06-a2bbc84be6de None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:43.072261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-58223446-fae9-48e9-bd06-a2bbc84be6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:43.176539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 686/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:05:43.231501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df647e5-076f-4c61-9895-b178a53f35b4 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:43.232725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df647e5-076f-4c61-9895-b178a53f35b4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:43.233399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df647e5-076f-4c61-9895-b178a53f35b4 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:43.235490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 672/1358] 10.222.0.47 () {54 vars in 962 bytes} [Wed Jul 27 20:05:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:43.244443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9d5f-c1cb-416a-9861-0e863cc30fe0 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:43.245324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9d5f-c1cb-416a-9861-0e863cc30fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:43.245864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9d5f-c1cb-416a-9861-0e863cc30fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:43.333143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4e4f9d5f-c1cb-416a-9861-0e863cc30fe0 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:43.334722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4e4f9d5f-c1cb-416a-9861-0e863cc30fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:43.392945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:43.470270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 687/1359] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Jul 27 20:05:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:05:43.472652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df647e5-076f-4c61-9895-b178a53f35b4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:43.474213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4df647e5-076f-4c61-9895-b178a53f35b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:43.626934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df647e5-076f-4c61-9895-b178a53f35b4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:43.699697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:43.740398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ea887-2a54-421c-a999-28d5f40b638f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:43.741015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ea887-2a54-421c-a999-28d5f40b638f service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:43.741293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ea887-2a54-421c-a999-28d5f40b638f service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:43.741562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ea887-2a54-421c-a999-28d5f40b638f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:43.743236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-322ea887-2a54-421c-a999-28d5f40b638f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:43.854420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ea887-2a54-421c-a999-28d5f40b638f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:43.856639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ea887-2a54-421c-a999-28d5f40b638f service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:43.878186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 673/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:05:46.119699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb4ef-5e1d-4eb9-9f75-819ffd37f699 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:46.120205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb4ef-5e1d-4eb9-9f75-819ffd37f699 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:46.120287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb4ef-5e1d-4eb9-9f75-819ffd37f699 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:46.125627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 688/1361] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:46.127099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42df86-4be7-4291-80fc-9c365b88e2a2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:46.127545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42df86-4be7-4291-80fc-9c365b88e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:46.127652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42df86-4be7-4291-80fc-9c365b88e2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:46.205804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8a42df86-4be7-4291-80fc-9c365b88e2a2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:46.206936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8a42df86-4be7-4291-80fc-9c365b88e2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:46.249797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:46.314527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 674/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:46.365783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bb4ef-5e1d-4eb9-9f75-819ffd37f699 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:46.367390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9a9bb4ef-5e1d-4eb9-9f75-819ffd37f699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:46.534060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bb4ef-5e1d-4eb9-9f75-819ffd37f699 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:46.545683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:46.682794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dafc43-ded2-4dc6-b5b1-e9c2398cbcea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:46.683508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dafc43-ded2-4dc6-b5b1-e9c2398cbcea service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:46.683854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dafc43-ded2-4dc6-b5b1-e9c2398cbcea service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:46.684191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dafc43-ded2-4dc6-b5b1-e9c2398cbcea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:46.686450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a0dafc43-ded2-4dc6-b5b1-e9c2398cbcea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:46.782206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dafc43-ded2-4dc6-b5b1-e9c2398cbcea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:46.784115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dafc43-ded2-4dc6-b5b1-e9c2398cbcea service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:46.804791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 689/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:49.319187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df005cb-7feb-4d5c-bbce-442ba0acab20 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:49.319697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df005cb-7feb-4d5c-bbce-442ba0acab20 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:49.319697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df005cb-7feb-4d5c-bbce-442ba0acab20 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:49.321879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 675/1364] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:49.327097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f7261-ed12-49ca-9424-0fcf712ea3d4 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:49.327578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f7261-ed12-49ca-9424-0fcf712ea3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:49.327662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f7261-ed12-49ca-9424-0fcf712ea3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:49.345892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:49.402273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f35f7261-ed12-49ca-9424-0fcf712ea3d4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:49.403488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f35f7261-ed12-49ca-9424-0fcf712ea3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:49.515663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 690/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:49.563472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d430a92-a794-44c5-b53c-e860e4acc189 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:49.563847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d430a92-a794-44c5-b53c-e860e4acc189 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:49.564232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d430a92-a794-44c5-b53c-e860e4acc189 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:49.565846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 676/1366] 10.222.0.47 () {54 vars in 962 bytes} [Wed Jul 27 20:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:49.571986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dd9f9-d4f5-4964-9393-f629adfe1c68 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:49.572206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dd9f9-d4f5-4964-9393-f629adfe1c68 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:49.572512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dd9f9-d4f5-4964-9393-f629adfe1c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:49.658089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-612dd9f9-d4f5-4964-9393-f629adfe1c68 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:49.659336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-612dd9f9-d4f5-4964-9393-f629adfe1c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:49.698330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:49.790671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 691/1367] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Jul 27 20:05:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:49.798694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d430a92-a794-44c5-b53c-e860e4acc189 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:49.800255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6d430a92-a794-44c5-b53c-e860e4acc189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:49.961508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d430a92-a794-44c5-b53c-e860e4acc189 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:49.983766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:50.083579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec25e4-ee4a-4817-988d-176895756da7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:50.084207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec25e4-ee4a-4817-988d-176895756da7 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:50.084644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec25e4-ee4a-4817-988d-176895756da7 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:50.084927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec25e4-ee4a-4817-988d-176895756da7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:50.086716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-90ec25e4-ee4a-4817-988d-176895756da7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:50.184959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec25e4-ee4a-4817-988d-176895756da7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:50.186809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec25e4-ee4a-4817-988d-176895756da7 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:50.206456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 677/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:51.160370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad72aa-ed38-4f4b-b52d-be63bc322348 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:51.160934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad72aa-ed38-4f4b-b52d-be63bc322348 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:51.161039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad72aa-ed38-4f4b-b52d-be63bc322348 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:51.162495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 692/1369] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:05:51.167674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee18b-529a-41f4-acba-a6a850f35bb2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:51.168091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee18b-529a-41f4-acba-a6a850f35bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:51.168286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee18b-529a-41f4-acba-a6a850f35bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:51.206129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:51.241585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-1fcee18b-529a-41f4-acba-a6a850f35bb2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:51.243435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1fcee18b-529a-41f4-acba-a6a850f35bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:51.348179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 678/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:51.696675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986a68-67e4-4155-80fe-08dd9d507aef None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:51.696844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986a68-67e4-4155-80fe-08dd9d507aef None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:51.697180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0986a68-67e4-4155-80fe-08dd9d507aef None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:51.698768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 693/1371] 10.222.0.47 () {54 vars in 964 bytes} [Wed Jul 27 20:05:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:51.703773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700b47-168f-42c3-aa9f-84e9c931ec77 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:51.704138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700b47-168f-42c3-aa9f-84e9c931ec77 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:51.704514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf700b47-168f-42c3-aa9f-84e9c931ec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:51.783821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-cf700b47-168f-42c3-aa9f-84e9c931ec77 None None] MFA Rules not processed for user `78776208cde94134b478ceee6cc70ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:51.785080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cf700b47-168f-42c3-aa9f-84e9c931ec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:51.833595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:51.864167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 679/1372] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Jul 27 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:05:51.871809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986a68-67e4-4155-80fe-08dd9d507aef None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:51.873461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a0986a68-67e4-4155-80fe-08dd9d507aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:51.985700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0986a68-67e4-4155-80fe-08dd9d507aef None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:52.028148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:52.063194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e366e1-0667-47a0-880a-f805b7a4aed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:52.064298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e366e1-0667-47a0-880a-f805b7a4aed8 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:52.064664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e366e1-0667-47a0-880a-f805b7a4aed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:52.064991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e366e1-0667-47a0-880a-f805b7a4aed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:52.068059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-10e366e1-0667-47a0-880a-f805b7a4aed8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:52.189422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e366e1-0667-47a0-880a-f805b7a4aed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:52.190875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e366e1-0667-47a0-880a-f805b7a4aed8 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:52.210175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 694/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:52.475901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2db52b-21bd-4a7e-a2ea-61e43168af0d None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:52.476342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2db52b-21bd-4a7e-a2ea-61e43168af0d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:52.476835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2db52b-21bd-4a7e-a2ea-61e43168af0d None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:52.479273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 680/1374] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:52.484731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ff166-9e6d-4c99-9b77-3296c3e77038 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:52.485139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ff166-9e6d-4c99-9b77-3296c3e77038 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:52.485517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ff166-9e6d-4c99-9b77-3296c3e77038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:52.565696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-454ff166-9e6d-4c99-9b77-3296c3e77038 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:52.567967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-454ff166-9e6d-4c99-9b77-3296c3e77038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:52.595553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:52.681794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 695/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:52.725659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2db52b-21bd-4a7e-a2ea-61e43168af0d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:52.727260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3d2db52b-21bd-4a7e-a2ea-61e43168af0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:52.872970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2db52b-21bd-4a7e-a2ea-61e43168af0d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:52.923837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:52.986466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a74cb6-9d28-4de9-a28d-93cd2f32d731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:52.987083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74cb6-9d28-4de9-a28d-93cd2f32d731 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:52.987361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74cb6-9d28-4de9-a28d-93cd2f32d731 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:52.987630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74cb6-9d28-4de9-a28d-93cd2f32d731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:52.989265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-83a74cb6-9d28-4de9-a28d-93cd2f32d731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:53.077152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a74cb6-9d28-4de9-a28d-93cd2f32d731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:53.078780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a74cb6-9d28-4de9-a28d-93cd2f32d731 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:53.097244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 681/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:55.242943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6bae9-3c94-42e5-8967-a61ca56685fd None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:55.243422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6bae9-3c94-42e5-8967-a61ca56685fd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:55.243519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6bae9-3c94-42e5-8967-a61ca56685fd None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:55.245484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 696/1377] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:55.251992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93700960-b925-49ec-b74a-a0de242a3ee7 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:55.252451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93700960-b925-49ec-b74a-a0de242a3ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:55.252582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93700960-b925-49ec-b74a-a0de242a3ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:55.332245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-93700960-b925-49ec-b74a-a0de242a3ee7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:55.333695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-93700960-b925-49ec-b74a-a0de242a3ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:55.341810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:55.432447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 682/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:55.475924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6bae9-3c94-42e5-8967-a61ca56685fd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:55.478070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e3e6bae9-3c94-42e5-8967-a61ca56685fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:55.633965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6bae9-3c94-42e5-8967-a61ca56685fd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:55.640314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:55.788845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156538a-1b82-451f-bc63-6a0dadbe0e84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:55.790474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156538a-1b82-451f-bc63-6a0dadbe0e84 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:55.790806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156538a-1b82-451f-bc63-6a0dadbe0e84 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:55.791087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156538a-1b82-451f-bc63-6a0dadbe0e84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:55.794405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a156538a-1b82-451f-bc63-6a0dadbe0e84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:55.905007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156538a-1b82-451f-bc63-6a0dadbe0e84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:55.906667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156538a-1b82-451f-bc63-6a0dadbe0e84 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:55.927127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 697/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:05:58.111874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fd680-d44f-4c09-b4f1-50881b5f6aa1 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:58.112426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fd680-d44f-4c09-b4f1-50881b5f6aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:58.112598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fd680-d44f-4c09-b4f1-50881b5f6aa1 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:58.113982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 683/1380] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:05:58.119111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1457-12f4-4e5a-a98c-f5c6d4be20de None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:58.119591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1457-12f4-4e5a-a98c-f5c6d4be20de None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:58.119741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f1457-12f4-4e5a-a98c-f5c6d4be20de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:58.131292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:58.189548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b47f1457-12f4-4e5a-a98c-f5c6d4be20de None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:05:58.190769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b47f1457-12f4-4e5a-a98c-f5c6d4be20de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:58.293604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 698/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:05:58.341879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fd680-d44f-4c09-b4f1-50881b5f6aa1 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:05:58.343484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-056fd680-d44f-4c09-b4f1-50881b5f6aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:58.495993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:58.507663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fd680-d44f-4c09-b4f1-50881b5f6aa1 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:05:58.650864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad4f78-9513-46fa-b36d-0b7620548fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:05:58.651573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4f78-9513-46fa-b36d-0b7620548fe8 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:05:58.651937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4f78-9513-46fa-b36d-0b7620548fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:05:58.652372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad4f78-9513-46fa-b36d-0b7620548fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:05:58.654201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fcad4f78-9513-46fa-b36d-0b7620548fe8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:05:58.756333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad4f78-9513-46fa-b36d-0b7620548fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:05:58.757974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad4f78-9513-46fa-b36d-0b7620548fe8 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:05:58.768848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:05:58.778820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 684/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:01.238417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b9e5-e40b-452d-9dcd-73ed6754bf2c None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:01.238978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b9e5-e40b-452d-9dcd-73ed6754bf2c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:01.239125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57b9e5-e40b-452d-9dcd-73ed6754bf2c None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:01.240601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 699/1383] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:06:01.245783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d334e-148a-43fb-b6b1-8e5e29941f7b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:01.246241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d334e-148a-43fb-b6b1-8e5e29941f7b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:01.246344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d334e-148a-43fb-b6b1-8e5e29941f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:01.314798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-0b9d334e-148a-43fb-b6b1-8e5e29941f7b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:01.316003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0b9d334e-148a-43fb-b6b1-8e5e29941f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:01.426977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 685/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:01.486086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57b9e5-e40b-452d-9dcd-73ed6754bf2c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:01.487868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-de57b9e5-e40b-452d-9dcd-73ed6754bf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:01.584516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:01.655273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57b9e5-e40b-452d-9dcd-73ed6754bf2c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:01.792422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb27178-8ed8-4e35-9a10-18624c604239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:01.793039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb27178-8ed8-4e35-9a10-18624c604239 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:01.793316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb27178-8ed8-4e35-9a10-18624c604239 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:01.793595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb27178-8ed8-4e35-9a10-18624c604239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:01.795211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2bb27178-8ed8-4e35-9a10-18624c604239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:01.844859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:01.886259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb27178-8ed8-4e35-9a10-18624c604239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:01.887874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb27178-8ed8-4e35-9a10-18624c604239 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:01.905890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 700/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:04.035697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac01d0c-07d9-430e-a0c1-53b401c1b1a6 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:04.036236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac01d0c-07d9-430e-a0c1-53b401c1b1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:04.036402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac01d0c-07d9-430e-a0c1-53b401c1b1a6 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:04.038259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 686/1386] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:04.044591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5363b9-b644-4f14-a07d-3b5d3c0d0218 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:04.045040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5363b9-b644-4f14-a07d-3b5d3c0d0218 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:04.045136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5363b9-b644-4f14-a07d-3b5d3c0d0218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:04.113804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fc5363b9-b644-4f14-a07d-3b5d3c0d0218 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:04.115029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fc5363b9-b644-4f14-a07d-3b5d3c0d0218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:04.241842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 701/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:04.288161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac01d0c-07d9-430e-a0c1-53b401c1b1a6 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:04.289875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1ac01d0c-07d9-430e-a0c1-53b401c1b1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:04.377357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:04.452235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac01d0c-07d9-430e-a0c1-53b401c1b1a6 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:04.573815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cc1ba-c52b-41f4-9b2c-13d946f66891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:04.574410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc1ba-c52b-41f4-9b2c-13d946f66891 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:04.574806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc1ba-c52b-41f4-9b2c-13d946f66891 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:04.575766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc1ba-c52b-41f4-9b2c-13d946f66891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:04.576853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7c7cc1ba-c52b-41f4-9b2c-13d946f66891 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:04.618373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:04.672321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cc1ba-c52b-41f4-9b2c-13d946f66891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:04.674501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cc1ba-c52b-41f4-9b2c-13d946f66891 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:04.699131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 687/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:06.952306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b00c8-f698-4000-bbd0-072574a062ff None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:06.952893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b00c8-f698-4000-bbd0-072574a062ff None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:06.952893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b00c8-f698-4000-bbd0-072574a062ff None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:06.954210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 702/1389] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:06.959315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a029a-8cbf-4e2c-90a8-9e6eba65dc8d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:06.959757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a029a-8cbf-4e2c-90a8-9e6eba65dc8d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:06.959871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a029a-8cbf-4e2c-90a8-9e6eba65dc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:07.028334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-fa7a029a-8cbf-4e2c-90a8-9e6eba65dc8d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:07.029571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fa7a029a-8cbf-4e2c-90a8-9e6eba65dc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:07.124703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 688/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:07.168889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b00c8-f698-4000-bbd0-072574a062ff None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:07.170450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-920b00c8-f698-4000-bbd0-072574a062ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:07.202847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:07.319394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b00c8-f698-4000-bbd0-072574a062ff None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:07.424975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:07.437356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71ebc0-5af9-4dd7-a0b1-d4039302df09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:07.452141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ebc0-5af9-4dd7-a0b1-d4039302df09 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:07.452141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ebc0-5af9-4dd7-a0b1-d4039302df09 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:07.452141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71ebc0-5af9-4dd7-a0b1-d4039302df09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:07.452141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2a71ebc0-5af9-4dd7-a0b1-d4039302df09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:07.529772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71ebc0-5af9-4dd7-a0b1-d4039302df09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:07.531358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71ebc0-5af9-4dd7-a0b1-d4039302df09 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:07.549117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 703/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:07.585546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce2e6-1795-4db9-bd2a-013726f27d1a None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:07.585838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce2e6-1795-4db9-bd2a-013726f27d1a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:07.586121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce2e6-1795-4db9-bd2a-013726f27d1a None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:07.587506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 689/1392] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:07.592894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbe8a-ac32-4201-adc6-8e1912eb8f21 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:07.593185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbe8a-ac32-4201-adc6-8e1912eb8f21 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:07.593461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbe8a-ac32-4201-adc6-8e1912eb8f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:07.659699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1d7dbe8a-ac32-4201-adc6-8e1912eb8f21 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:07.660946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1d7dbe8a-ac32-4201-adc6-8e1912eb8f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:07.721573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:07.759971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 704/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:07.817588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eb8c6-f791-4666-a6d5-7b9a69cb9ac0 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:07.817873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eb8c6-f791-4666-a6d5-7b9a69cb9ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:07.818150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eb8c6-f791-4666-a6d5-7b9a69cb9ac0 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:07.819541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 690/1394] 10.222.0.47 () {54 vars in 966 bytes} [Wed Jul 27 20:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:07.825640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799289d-bf9b-4c42-aba7-fa2ee69bdc77 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:07.825928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799289d-bf9b-4c42-aba7-fa2ee69bdc77 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:07.826206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799289d-bf9b-4c42-aba7-fa2ee69bdc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:07.914614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c799289d-bf9b-4c42-aba7-fa2ee69bdc77 None None] MFA Rules not processed for user `813427e9e85a4b48a550a290211f39b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:07.915915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c799289d-bf9b-4c42-aba7-fa2ee69bdc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:07.994955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 705/1395] 10.222.0.47 () {58 vars in 1063 bytes} [Wed Jul 27 20:06:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:08.002884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080eb8c6-f791-4666-a6d5-7b9a69cb9ac0 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:08.004459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-080eb8c6-f791-4666-a6d5-7b9a69cb9ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:08.018577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:08.110878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080eb8c6-f791-4666-a6d5-7b9a69cb9ac0 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:08.187859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4e8b2-fca4-4051-ad38-4e17157c0cbb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:08.188500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4e8b2-fca4-4051-ad38-4e17157c0cbb service glance] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:08.188784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4e8b2-fca4-4051-ad38-4e17157c0cbb service glance] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:08.189079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4e8b2-fca4-4051-ad38-4e17157c0cbb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:08.190691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6fd4e8b2-fca4-4051-ad38-4e17157c0cbb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:08.260412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:08.301019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4e8b2-fca4-4051-ad38-4e17157c0cbb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:08.302457 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4e8b2-fca4-4051-ad38-4e17157c0cbb service glance] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:08.321415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 691/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:09.198192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39ec0-5481-4001-8e55-36419afaa328 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:09.198675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39ec0-5481-4001-8e55-36419afaa328 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:09.198776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39ec0-5481-4001-8e55-36419afaa328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:09.266615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-00b39ec0-5481-4001-8e55-36419afaa328 None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:09.267837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-00b39ec0-5481-4001-8e55-36419afaa328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:09.276408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:09.359132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 706/1397] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:06:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:09.404960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74da793-6231-4b3a-a592-d85dd3087e2e None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:09.405468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74da793-6231-4b3a-a592-d85dd3087e2e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:09.405569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74da793-6231-4b3a-a592-d85dd3087e2e None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:09.406915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 692/1398] 10.222.0.47 () {54 vars in 943 bytes} [Wed Jul 27 20:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:09.416206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908c22f-7cb1-4f44-b5d5-08f04955182d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:09.416526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908c22f-7cb1-4f44-b5d5-08f04955182d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:09.416804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908c22f-7cb1-4f44-b5d5-08f04955182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:09.496656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3908c22f-7cb1-4f44-b5d5-08f04955182d None None] MFA Rules not processed for user `dcebfbf0b3084929bc33887467695b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:09.497862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3908c22f-7cb1-4f44-b5d5-08f04955182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:09.589112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:09.735270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 707/1399] 10.222.0.47 () {58 vars in 1040 bytes} [Wed Jul 27 20:06:09 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:09.745566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74da793-6231-4b3a-a592-d85dd3087e2e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:09.747145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c74da793-6231-4b3a-a592-d85dd3087e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:09.776592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feb4f5-3d72-4433-9b41-f133e64106af None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:09.776875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feb4f5-3d72-4433-9b41-f133e64106af None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:09.777146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4feb4f5-3d72-4433-9b41-f133e64106af None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:09.778491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 708/1400] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:06:09.783320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:09.783599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:09.783902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:09.849457 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:09.850942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:09.888091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74da793-6231-4b3a-a592-d85dd3087e2e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:09.924585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:09.950673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 709/1401] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:06:09.990535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae0034-a621-4b0c-a409-da9ab40ed6f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:09.991158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0034-a621-4b0c-a409-da9ab40ed6f8 service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:09.991437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0034-a621-4b0c-a409-da9ab40ed6f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:09.991712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae0034-a621-4b0c-a409-da9ab40ed6f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:09.993932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-47ae0034-a621-4b0c-a409-da9ab40ed6f8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:09.995413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:09.996623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:10.061445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:10.078075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae0034-a621-4b0c-a409-da9ab40ed6f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:10.079675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae0034-a621-4b0c-a409-da9ab40ed6f8 service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:10.088338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 693/1402] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:06:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:06:10.114058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e084e5-4af9-4b1f-bcfe-bad76f850fda None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:10.226233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10dadf-7fc3-4324-b830-2b954d64abb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:10.226890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10dadf-7fc3-4324-b830-2b954d64abb7 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:10.227160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10dadf-7fc3-4324-b830-2b954d64abb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:10.227450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10dadf-7fc3-4324-b830-2b954d64abb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:10.229155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1d10dadf-7fc3-4324-b830-2b954d64abb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:10.287840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:10.316963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10dadf-7fc3-4324-b830-2b954d64abb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:10.318511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10dadf-7fc3-4324-b830-2b954d64abb7 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:10.337700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 710/1403] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:06:12.580293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b71b-412d-4f38-8e6c-a3e03aa9463f None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:12.580895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b71b-412d-4f38-8e6c-a3e03aa9463f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:12.580895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b71b-412d-4f38-8e6c-a3e03aa9463f None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:12.582289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 694/1404] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:12.588232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2d19-5d86-48e6-8bd9-568b3627b76d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:12.595686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2d19-5d86-48e6-8bd9-568b3627b76d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:12.595686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2d19-5d86-48e6-8bd9-568b3627b76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:12.662673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-639e2d19-5d86-48e6-8bd9-568b3627b76d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:12.664850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-639e2d19-5d86-48e6-8bd9-568b3627b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:12.770558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 711/1405] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:12.816336 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0b71b-412d-4f38-8e6c-a3e03aa9463f None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:12.818114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8ec0b71b-412d-4f38-8e6c-a3e03aa9463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:12.901902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:12.988105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0b71b-412d-4f38-8e6c-a3e03aa9463f None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:13.118256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea5cae-0f07-4a93-9444-ad7339a0001a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:13.119191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5cae-0f07-4a93-9444-ad7339a0001a service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:13.119477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5cae-0f07-4a93-9444-ad7339a0001a service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:13.119758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5cae-0f07-4a93-9444-ad7339a0001a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:13.121450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-afea5cae-0f07-4a93-9444-ad7339a0001a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:13.159081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:13.228458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea5cae-0f07-4a93-9444-ad7339a0001a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:13.230203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea5cae-0f07-4a93-9444-ad7339a0001a service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:13.249248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 695/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:15.470117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d3866-3e71-490e-a393-b6889f6b0758 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:15.470673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d3866-3e71-490e-a393-b6889f6b0758 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:15.470673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d3866-3e71-490e-a393-b6889f6b0758 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:15.472019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 712/1407] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:15.477279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3251d6-4e50-48c2-9f4c-58ad349f2687 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:15.478205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3251d6-4e50-48c2-9f4c-58ad349f2687 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:15.478704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3251d6-4e50-48c2-9f4c-58ad349f2687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:15.548622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-dd3251d6-4e50-48c2-9f4c-58ad349f2687 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:15.549942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-dd3251d6-4e50-48c2-9f4c-58ad349f2687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:15.667649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 696/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:15.714249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d3866-3e71-490e-a393-b6889f6b0758 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:15.718194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a55d3866-3e71-490e-a393-b6889f6b0758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:15.762574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:15.887722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d3866-3e71-490e-a393-b6889f6b0758 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:16.018759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:16.024935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c84d2-8c4f-4b5b-a5c5-688854a03129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:16.025564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c84d2-8c4f-4b5b-a5c5-688854a03129 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:16.025842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c84d2-8c4f-4b5b-a5c5-688854a03129 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:16.026182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c84d2-8c4f-4b5b-a5c5-688854a03129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:16.027995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b68c84d2-8c4f-4b5b-a5c5-688854a03129 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:16.143102 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c84d2-8c4f-4b5b-a5c5-688854a03129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:16.144875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c84d2-8c4f-4b5b-a5c5-688854a03129 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:16.165758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 713/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:25.665815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:26.162163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=47 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:34.070504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64d88-1ee7-49d8-b4b7-a2e19ac36f06 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:34.071047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64d88-1ee7-49d8-b4b7-a2e19ac36f06 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:34.071047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64d88-1ee7-49d8-b4b7-a2e19ac36f06 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:34.072568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 697/1410] 10.222.0.47 () {54 vars in 918 bytes} [Wed Jul 27 20:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:34.079482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1e65-5f6e-49c4-859b-546665d1f899 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:34.079925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1e65-5f6e-49c4-859b-546665d1f899 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:34.080037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b1e65-5f6e-49c4-859b-546665d1f899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:34.154909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7d8b1e65-5f6e-49c4-859b-546665d1f899 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:34.156120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7d8b1e65-5f6e-49c4-859b-546665d1f899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:34.258740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 714/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Jul 27 20:06:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:34.273621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4278908-d2a3-4225-93d3-edfcb79daf2b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:34.273905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4278908-d2a3-4225-93d3-edfcb79daf2b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:34.274179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4278908-d2a3-4225-93d3-edfcb79daf2b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:34.275533 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 698/1412] 10.222.0.47 () {54 vars in 918 bytes} [Wed Jul 27 20:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:34.280172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13045a-abe4-4100-8986-e9e320ffde54 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:34.280488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13045a-abe4-4100-8986-e9e320ffde54 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:34.280767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13045a-abe4-4100-8986-e9e320ffde54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:34.351748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:34.354769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1d13045a-abe4-4100-8986-e9e320ffde54 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:34.355970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1d13045a-abe4-4100-8986-e9e320ffde54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:34.471731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 715/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Jul 27 20:06:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:36.959467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11146d-33f1-4645-9468-8ec711b1364e None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:36.959976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11146d-33f1-4645-9468-8ec711b1364e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:36.959976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11146d-33f1-4645-9468-8ec711b1364e None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:36.961882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:36.967147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc920721-f37e-47f4-bf9e-205e9ff7f600 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:36.967535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc920721-f37e-47f4-bf9e-205e9ff7f600 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:36.967693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc920721-f37e-47f4-bf9e-205e9ff7f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:37.034677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-dc920721-f37e-47f4-bf9e-205e9ff7f600 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:37.035888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dc920721-f37e-47f4-bf9e-205e9ff7f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:37.091648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:37.133168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:06:37.175848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11146d-33f1-4645-9468-8ec711b1364e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:37.177429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0a11146d-33f1-4645-9468-8ec711b1364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:37.322455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11146d-33f1-4645-9468-8ec711b1364e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:37.396244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:37.436065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394401de-db38-4aba-ab8d-0fa8539e974b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:37.436714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394401de-db38-4aba-ab8d-0fa8539e974b service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:37.436995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394401de-db38-4aba-ab8d-0fa8539e974b service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:37.437264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394401de-db38-4aba-ab8d-0fa8539e974b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:37.438870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-394401de-db38-4aba-ab8d-0fa8539e974b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:37.526026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394401de-db38-4aba-ab8d-0fa8539e974b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:37.527654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394401de-db38-4aba-ab8d-0fa8539e974b service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:37.545432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 700/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:38.438176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995bd05-3dfe-4b37-82bf-fbe9ba9fb9bc None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:38.438701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995bd05-3dfe-4b37-82bf-fbe9ba9fb9bc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:38.438701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995bd05-3dfe-4b37-82bf-fbe9ba9fb9bc None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:38.440052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:38.445469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda6479-80fb-44ad-a9bc-f16ce214da01 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:38.445864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda6479-80fb-44ad-a9bc-f16ce214da01 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:38.446026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda6479-80fb-44ad-a9bc-f16ce214da01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:38.520977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7eda6479-80fb-44ad-a9bc-f16ce214da01 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:38.522216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7eda6479-80fb-44ad-a9bc-f16ce214da01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:38.549083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:38.620705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:38.772021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995bd05-3dfe-4b37-82bf-fbe9ba9fb9bc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:38.774186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1995bd05-3dfe-4b37-82bf-fbe9ba9fb9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:38.918616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995bd05-3dfe-4b37-82bf-fbe9ba9fb9bc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:38.957973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:39.028775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a7744-ea8f-4c92-8e9c-38919a7c26a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:39.029394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a7744-ea8f-4c92-8e9c-38919a7c26a1 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:39.029672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a7744-ea8f-4c92-8e9c-38919a7c26a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:39.029945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a7744-ea8f-4c92-8e9c-38919a7c26a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:39.031589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-789a7744-ea8f-4c92-8e9c-38919a7c26a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:39.119421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a7744-ea8f-4c92-8e9c-38919a7c26a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:39.121043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a7744-ea8f-4c92-8e9c-38919a7c26a1 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:39.139125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 718/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:40.022088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939215-1a2e-4d27-bed6-c63a23205c51 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:40.022732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939215-1a2e-4d27-bed6-c63a23205c51 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:40.022732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939215-1a2e-4d27-bed6-c63a23205c51 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:40.024026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:40.028934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8704b-6251-45c8-8fed-6306370bb247 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:40.029377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8704b-6251-45c8-8fed-6306370bb247 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:40.029474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8704b-6251-45c8-8fed-6306370bb247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:40.083645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:40.106089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-56f8704b-6251-45c8-8fed-6306370bb247 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:40.113937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-56f8704b-6251-45c8-8fed-6306370bb247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:40.203528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:40.354888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8939215-1a2e-4d27-bed6-c63a23205c51 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:40.356545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e8939215-1a2e-4d27-bed6-c63a23205c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:40.507716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8939215-1a2e-4d27-bed6-c63a23205c51 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:40.519751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:40.619131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff8ade-7059-44d4-bca8-5255ff749f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:40.619743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff8ade-7059-44d4-bca8-5255ff749f7b service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:40.620019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff8ade-7059-44d4-bca8-5255ff749f7b service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:40.620287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff8ade-7059-44d4-bca8-5255ff749f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:40.621912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-89ff8ade-7059-44d4-bca8-5255ff749f7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:40.709329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff8ade-7059-44d4-bca8-5255ff749f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:40.711018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff8ade-7059-44d4-bca8-5255ff749f7b service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:40.728461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 703/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:41.592267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a1e5a-44cf-474d-89e4-9c49e6431ab1 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:41.592824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a1e5a-44cf-474d-89e4-9c49e6431ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:41.592824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a1e5a-44cf-474d-89e4-9c49e6431ab1 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:41.594144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:41.599121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010625-265e-41ed-8b6c-f22fb9be7ed2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:41.599551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010625-265e-41ed-8b6c-f22fb9be7ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:41.599658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9010625-265e-41ed-8b6c-f22fb9be7ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:41.619401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:41.678039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a9010625-265e-41ed-8b6c-f22fb9be7ed2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:41.679243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a9010625-265e-41ed-8b6c-f22fb9be7ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:41.773776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:41.923840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a1e5a-44cf-474d-89e4-9c49e6431ab1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:41.925330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e48a1e5a-44cf-474d-89e4-9c49e6431ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:42.032935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:42.068453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a1e5a-44cf-474d-89e4-9c49e6431ab1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:42.178755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c6bac-a32f-4e5b-8ea6-bcfc12333129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:42.179299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c6bac-a32f-4e5b-8ea6-bcfc12333129 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:42.179577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c6bac-a32f-4e5b-8ea6-bcfc12333129 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:42.179849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c6bac-a32f-4e5b-8ea6-bcfc12333129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:42.182060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-008c6bac-a32f-4e5b-8ea6-bcfc12333129 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:42.256844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:42.269887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c6bac-a32f-4e5b-8ea6-bcfc12333129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:42.271470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c6bac-a32f-4e5b-8ea6-bcfc12333129 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:42.289719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 721/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:43.155895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72644f85-e78f-4996-9bf3-cd0b95231c04 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:43.156495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72644f85-e78f-4996-9bf3-cd0b95231c04 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:43.156495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72644f85-e78f-4996-9bf3-cd0b95231c04 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:43.157837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:43.162586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38efbe-4048-4698-9962-f485f73356c4 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:43.163004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38efbe-4048-4698-9962-f485f73356c4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:43.163130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38efbe-4048-4698-9962-f485f73356c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:43.238530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6d38efbe-4048-4698-9962-f485f73356c4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:43.239731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6d38efbe-4048-4698-9962-f485f73356c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:43.333512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:43.485689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72644f85-e78f-4996-9bf3-cd0b95231c04 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:43.487465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-72644f85-e78f-4996-9bf3-cd0b95231c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:43.570195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:43.632271 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72644f85-e78f-4996-9bf3-cd0b95231c04 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:43.743955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5f170-5da4-4b43-9aa7-fd7b4cff0e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:43.744520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5f170-5da4-4b43-9aa7-fd7b4cff0e51 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:43.744805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5f170-5da4-4b43-9aa7-fd7b4cff0e51 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:43.745078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5f170-5da4-4b43-9aa7-fd7b4cff0e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:43.746703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-95c5f170-5da4-4b43-9aa7-fd7b4cff0e51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:43.795357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:43.835778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5f170-5da4-4b43-9aa7-fd7b4cff0e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:43.837997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5f170-5da4-4b43-9aa7-fd7b4cff0e51 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:43.855724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 706/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:44.733803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f93c64-5c51-4dc9-8cc3-f0de440b327b None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:44.734330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f93c64-5c51-4dc9-8cc3-f0de440b327b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:44.734447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f93c64-5c51-4dc9-8cc3-f0de440b327b None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:44.735738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:44.740941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89b94-7845-40c4-b609-a29a371bd1d3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:44.741339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89b94-7845-40c4-b609-a29a371bd1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:44.741502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c89b94-7845-40c4-b609-a29a371bd1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:44.816785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-99c89b94-7845-40c4-b609-a29a371bd1d3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:44.818003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-99c89b94-7845-40c4-b609-a29a371bd1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:44.915096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:45.066468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f93c64-5c51-4dc9-8cc3-f0de440b327b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:45.067981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-89f93c64-5c51-4dc9-8cc3-f0de440b327b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:45.116323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:45.211807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f93c64-5c51-4dc9-8cc3-f0de440b327b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:45.322169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba065334-21cb-477c-82a5-ebe4a6151edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:45.322779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba065334-21cb-477c-82a5-ebe4a6151edd service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:45.323057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba065334-21cb-477c-82a5-ebe4a6151edd service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:45.323326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba065334-21cb-477c-82a5-ebe4a6151edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:45.324968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ba065334-21cb-477c-82a5-ebe4a6151edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:45.331908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:45.414750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba065334-21cb-477c-82a5-ebe4a6151edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:45.416867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba065334-21cb-477c-82a5-ebe4a6151edd service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:45.434918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 724/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:46.295782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2171f5-ea21-4ef8-a3f0-d7e30125b1f6 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:46.296417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2171f5-ea21-4ef8-a3f0-d7e30125b1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:46.296417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2171f5-ea21-4ef8-a3f0-d7e30125b1f6 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:46.297723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:46.302520 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10b146-d074-4871-a8be-404944e4df20 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:46.302938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10b146-d074-4871-a8be-404944e4df20 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:46.303066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10b146-d074-4871-a8be-404944e4df20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:46.380057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2e10b146-d074-4871-a8be-404944e4df20 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:46.381305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2e10b146-d074-4871-a8be-404944e4df20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:46.448066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:46.476947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:46.629622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2171f5-ea21-4ef8-a3f0-d7e30125b1f6 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:46.631146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ed2171f5-ea21-4ef8-a3f0-d7e30125b1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:46.648846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:46.779447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2171f5-ea21-4ef8-a3f0-d7e30125b1f6 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:46.868341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:46.893921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09e6e6-a939-4c2c-a919-939d1b578b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:46.894736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09e6e6-a939-4c2c-a919-939d1b578b82 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:46.895231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09e6e6-a939-4c2c-a919-939d1b578b82 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:46.895782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09e6e6-a939-4c2c-a919-939d1b578b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:46.897521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-be09e6e6-a939-4c2c-a919-939d1b578b82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:46.987001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09e6e6-a939-4c2c-a919-939d1b578b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:46.988620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09e6e6-a939-4c2c-a919-939d1b578b82 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:47.006195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 709/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:47.877042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ccaaf-734e-42a7-bfd9-377eeb05995d None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:47.877579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ccaaf-734e-42a7-bfd9-377eeb05995d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:47.877579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ccaaf-734e-42a7-bfd9-377eeb05995d None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:47.878946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:47.884353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0472-2c2b-4d33-aab0-5980631cfb2a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:47.884797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0472-2c2b-4d33-aab0-5980631cfb2a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:47.884938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0472-2c2b-4d33-aab0-5980631cfb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:47.960587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d22a0472-2c2b-4d33-aab0-5980631cfb2a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:47.961843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d22a0472-2c2b-4d33-aab0-5980631cfb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:48.005039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:48.059635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:48.209321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ccaaf-734e-42a7-bfd9-377eeb05995d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:48.210753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bf5ccaaf-734e-42a7-bfd9-377eeb05995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:48.354166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ccaaf-734e-42a7-bfd9-377eeb05995d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:48.406767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:48.465712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e499f-1ee7-468c-b914-ddbe30ceab33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:48.466260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e499f-1ee7-468c-b914-ddbe30ceab33 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:48.466539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e499f-1ee7-468c-b914-ddbe30ceab33 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:48.466809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e499f-1ee7-468c-b914-ddbe30ceab33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:48.468459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6c5e499f-1ee7-468c-b914-ddbe30ceab33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:48.557299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e499f-1ee7-468c-b914-ddbe30ceab33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:48.558879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e499f-1ee7-468c-b914-ddbe30ceab33 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:48.577485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 727/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:49.449098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49bda9-2d08-4009-a95a-e7f420f46bf1 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:49.449630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49bda9-2d08-4009-a95a-e7f420f46bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:49.449630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49bda9-2d08-4009-a95a-e7f420f46bf1 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:49.451064 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:49.455864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df6a2-e717-4167-b115-397cb9133bf2 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:49.456274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df6a2-e717-4167-b115-397cb9133bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:49.456451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df6a2-e717-4167-b115-397cb9133bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:49.536060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6b4df6a2-e717-4167-b115-397cb9133bf2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:49.537396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6b4df6a2-e717-4167-b115-397cb9133bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:49.539417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:49.634141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:49.786170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49bda9-2d08-4009-a95a-e7f420f46bf1 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:49.788318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fe49bda9-2d08-4009-a95a-e7f420f46bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:49.934963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49bda9-2d08-4009-a95a-e7f420f46bf1 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:49.960996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:50.049778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1ac66-f5c2-4bf9-a7b3-2dca020fda09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:50.050400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1ac66-f5c2-4bf9-a7b3-2dca020fda09 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:50.050678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1ac66-f5c2-4bf9-a7b3-2dca020fda09 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:50.050952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1ac66-f5c2-4bf9-a7b3-2dca020fda09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:50.052609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fff1ac66-f5c2-4bf9-a7b3-2dca020fda09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:50.140867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1ac66-f5c2-4bf9-a7b3-2dca020fda09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:50.142464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1ac66-f5c2-4bf9-a7b3-2dca020fda09 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:50.160134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 712/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:52.065465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d7d4-9dcc-4784-8c97-da64f573bc88 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:52.066034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d7d4-9dcc-4784-8c97-da64f573bc88 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:52.066034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d7d4-9dcc-4784-8c97-da64f573bc88 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:52.067431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:52.073757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38753815-1768-431b-a5ed-7245f3d0d209 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:52.074221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38753815-1768-431b-a5ed-7245f3d0d209 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:52.074334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38753815-1768-431b-a5ed-7245f3d0d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:52.116992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:52.142847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-38753815-1768-431b-a5ed-7245f3d0d209 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:52.144169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-38753815-1768-431b-a5ed-7245f3d0d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:52.240366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:52.279383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d7d4-9dcc-4784-8c97-da64f573bc88 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:52.280988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b542d7d4-9dcc-4784-8c97-da64f573bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:52.384174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d7d4-9dcc-4784-8c97-da64f573bc88 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:52.416895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:52.457401 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0e17e-965d-455c-bae8-099397efcdc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:52.458015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e17e-965d-455c-bae8-099397efcdc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:52.458294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e17e-965d-455c-bae8-099397efcdc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:52.458567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e17e-965d-455c-bae8-099397efcdc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:52.460191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fbb0e17e-965d-455c-bae8-099397efcdc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:52.565763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0e17e-965d-455c-bae8-099397efcdc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:52.567179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0e17e-965d-455c-bae8-099397efcdc6 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:52.585407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 730/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:54.769270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a12af-4320-4b7b-8613-bb8e9f0c626c None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:54.769880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a12af-4320-4b7b-8613-bb8e9f0c626c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:54.769880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a12af-4320-4b7b-8613-bb8e9f0c626c None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:54.771186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:54.776415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d355d-b506-4f6c-99fe-51d663a9a265 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:54.776850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d355d-b506-4f6c-99fe-51d663a9a265 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:54.776955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d355d-b506-4f6c-99fe-51d663a9a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:54.842673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-383d355d-b506-4f6c-99fe-51d663a9a265 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:54.843873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-383d355d-b506-4f6c-99fe-51d663a9a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:54.857151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:54.938564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:54.976956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a12af-4320-4b7b-8613-bb8e9f0c626c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:54.978611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-119a12af-4320-4b7b-8613-bb8e9f0c626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:55.083724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a12af-4320-4b7b-8613-bb8e9f0c626c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:55.096279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:55.156753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55415e1-3809-4c35-901f-7ea75e0e5af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:55.157299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55415e1-3809-4c35-901f-7ea75e0e5af3 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:55.157572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55415e1-3809-4c35-901f-7ea75e0e5af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:55.157845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55415e1-3809-4c35-901f-7ea75e0e5af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:55.159478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a55415e1-3809-4c35-901f-7ea75e0e5af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:55.270856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55415e1-3809-4c35-901f-7ea75e0e5af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:55.272316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55415e1-3809-4c35-901f-7ea75e0e5af3 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:55.290548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 715/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:57.663666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e635a0c-3ad8-44eb-965e-5fb56ac16e9a None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:57.664215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e635a0c-3ad8-44eb-965e-5fb56ac16e9a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:57.664215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e635a0c-3ad8-44eb-965e-5fb56ac16e9a None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:57.665540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:57.670684 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd88c3e-49f2-4406-9d2c-5a7216d86209 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:57.671067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd88c3e-49f2-4406-9d2c-5a7216d86209 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:57.671240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd88c3e-49f2-4406-9d2c-5a7216d86209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:57.719038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:57.735961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-2cd88c3e-49f2-4406-9d2c-5a7216d86209 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:57.737460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2cd88c3e-49f2-4406-9d2c-5a7216d86209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:57.834781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:57.881012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e635a0c-3ad8-44eb-965e-5fb56ac16e9a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:57.882545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1e635a0c-3ad8-44eb-965e-5fb56ac16e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:57.985089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e635a0c-3ad8-44eb-965e-5fb56ac16e9a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:58.025533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:58.059324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddeac5c-5f82-4ba0-8da5-15fa66b50fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:58.059939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeac5c-5f82-4ba0-8da5-15fa66b50fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:58.060214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeac5c-5f82-4ba0-8da5-15fa66b50fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:58.060516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeac5c-5f82-4ba0-8da5-15fa66b50fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:58.062137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6ddeac5c-5f82-4ba0-8da5-15fa66b50fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:58.169351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddeac5c-5f82-4ba0-8da5-15fa66b50fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:58.170808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddeac5c-5f82-4ba0-8da5-15fa66b50fe3 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:58.188215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 733/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:00.295211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b5946-47de-4154-9f21-8a170e1ef1af None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:00.295780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b5946-47de-4154-9f21-8a170e1ef1af None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:00.295780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b5946-47de-4154-9f21-8a170e1ef1af None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:00.297161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:00.302460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4933fa-df6c-47a9-a45f-206f9c7c2171 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:00.302891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4933fa-df6c-47a9-a45f-206f9c7c2171 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:00.302997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4933fa-df6c-47a9-a45f-206f9c7c2171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:00.368963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fa4933fa-df6c-47a9-a45f-206f9c7c2171 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:00.370163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fa4933fa-df6c-47a9-a45f-206f9c7c2171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:00.390632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:00.468461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:00.507702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b5946-47de-4154-9f21-8a170e1ef1af None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:00.509361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-db6b5946-47de-4154-9f21-8a170e1ef1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:00.611344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b5946-47de-4154-9f21-8a170e1ef1af None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:00.632007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:00.685398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b0fe7-42cc-438b-923c-31ca307d7c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:00.686013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b0fe7-42cc-438b-923c-31ca307d7c5e service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:00.686297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b0fe7-42cc-438b-923c-31ca307d7c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:00.686574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b0fe7-42cc-438b-923c-31ca307d7c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:00.688198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-628b0fe7-42cc-438b-923c-31ca307d7c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:00.797943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b0fe7-42cc-438b-923c-31ca307d7c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:00.799369 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b0fe7-42cc-438b-923c-31ca307d7c5e service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:00.817578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 718/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:02.860275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1e4b-87ec-4826-a722-b5a69a013ce4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:02.860821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1e4b-87ec-4826-a722-b5a69a013ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:02.860821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1e4b-87ec-4826-a722-b5a69a013ce4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:02.862436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:02.868188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b97f-904c-4eaf-8107-83dd4beec9e2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:02.868682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b97f-904c-4eaf-8107-83dd4beec9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:02.868791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014b97f-904c-4eaf-8107-83dd4beec9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:02.931943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:02.934615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3014b97f-904c-4eaf-8107-83dd4beec9e2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:02.935844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3014b97f-904c-4eaf-8107-83dd4beec9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:03.031717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:03.069765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec1e4b-87ec-4826-a722-b5a69a013ce4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:03.071437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6eec1e4b-87ec-4826-a722-b5a69a013ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:03.172407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec1e4b-87ec-4826-a722-b5a69a013ce4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:03.220521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:03.245791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65aa26-5700-4b03-a20e-e57cf7825cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:03.246397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65aa26-5700-4b03-a20e-e57cf7825cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:03.246672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65aa26-5700-4b03-a20e-e57cf7825cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:03.246939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65aa26-5700-4b03-a20e-e57cf7825cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:03.249245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dd65aa26-5700-4b03-a20e-e57cf7825cc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:03.354361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65aa26-5700-4b03-a20e-e57cf7825cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:03.355792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65aa26-5700-4b03-a20e-e57cf7825cc4 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:03.374648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 736/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:05.426019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b609b-ef9b-49b7-89d2-d2a00e48fd97 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:05.426578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b609b-ef9b-49b7-89d2-d2a00e48fd97 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:05.426578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b609b-ef9b-49b7-89d2-d2a00e48fd97 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:05.427925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:05.433790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec271992-0034-42ba-8908-64f689368eff None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:05.434224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec271992-0034-42ba-8908-64f689368eff None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:05.434337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec271992-0034-42ba-8908-64f689368eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:05.500353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ec271992-0034-42ba-8908-64f689368eff None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:05.501603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ec271992-0034-42ba-8908-64f689368eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:05.530388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:05.598916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:05.637661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b609b-ef9b-49b7-89d2-d2a00e48fd97 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:05.639217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c52b609b-ef9b-49b7-89d2-d2a00e48fd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:05.741799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b609b-ef9b-49b7-89d2-d2a00e48fd97 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:05.770187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:05.814890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78df67c-ba62-4ad7-ba4c-55637e0535f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:05.815511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78df67c-ba62-4ad7-ba4c-55637e0535f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:05.815797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78df67c-ba62-4ad7-ba4c-55637e0535f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:05.816071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78df67c-ba62-4ad7-ba4c-55637e0535f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:05.817738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a78df67c-ba62-4ad7-ba4c-55637e0535f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:05.926706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78df67c-ba62-4ad7-ba4c-55637e0535f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:05.928060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78df67c-ba62-4ad7-ba4c-55637e0535f4 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:05.947670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 721/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:08.019309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c041d6-00fd-4128-917b-d105e9d7f438 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:08.019847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c041d6-00fd-4128-917b-d105e9d7f438 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:08.019946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c041d6-00fd-4128-917b-d105e9d7f438 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:08.021267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:08.026550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81830007-5b7a-4aeb-a5ab-7bb2042a7b9c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:08.026997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81830007-5b7a-4aeb-a5ab-7bb2042a7b9c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:08.027102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81830007-5b7a-4aeb-a5ab-7bb2042a7b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:08.215526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-81830007-5b7a-4aeb-a5ab-7bb2042a7b9c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:08.216779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-81830007-5b7a-4aeb-a5ab-7bb2042a7b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:08.225212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:08.312646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:08.349596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c041d6-00fd-4128-917b-d105e9d7f438 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:08.351138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-18c041d6-00fd-4128-917b-d105e9d7f438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:08.454406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c041d6-00fd-4128-917b-d105e9d7f438 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:08.507821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:08.525203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0043f4-9e68-4d25-b7cc-d483125c5909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:08.525808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0043f4-9e68-4d25-b7cc-d483125c5909 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:08.526083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0043f4-9e68-4d25-b7cc-d483125c5909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:08.526346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0043f4-9e68-4d25-b7cc-d483125c5909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:08.528017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9d0043f4-9e68-4d25-b7cc-d483125c5909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:08.635043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0043f4-9e68-4d25-b7cc-d483125c5909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:08.636559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0043f4-9e68-4d25-b7cc-d483125c5909 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:08.655146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 739/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:10.687501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293dd12-862c-452a-a5e1-bf6f87123cc3 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:10.688043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293dd12-862c-452a-a5e1-bf6f87123cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:10.688043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293dd12-862c-452a-a5e1-bf6f87123cc3 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:10.689414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:10.694682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926c84c-f392-4b22-9620-78af22ab71ba None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:10.695061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926c84c-f392-4b22-9620-78af22ab71ba None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:10.695850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926c84c-f392-4b22-9620-78af22ab71ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:10.760846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9926c84c-f392-4b22-9620-78af22ab71ba None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:10.762068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9926c84c-f392-4b22-9620-78af22ab71ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:10.799876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:10.858154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:10.895487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293dd12-862c-452a-a5e1-bf6f87123cc3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:10.897045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9293dd12-862c-452a-a5e1-bf6f87123cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:10.998307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293dd12-862c-452a-a5e1-bf6f87123cc3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:11.034258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:11.071494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b8032-3112-4058-a04f-021fc3c18556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:11.072105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8032-3112-4058-a04f-021fc3c18556 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:11.072398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8032-3112-4058-a04f-021fc3c18556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:11.072672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b8032-3112-4058-a04f-021fc3c18556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:11.074295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-637b8032-3112-4058-a04f-021fc3c18556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:11.181073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b8032-3112-4058-a04f-021fc3c18556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:11.182413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b8032-3112-4058-a04f-021fc3c18556 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:11.200432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 724/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:07:13.226126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53510a7-59ed-4537-b4d1-31baf63a5466 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:13.226702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53510a7-59ed-4537-b4d1-31baf63a5466 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:13.226903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53510a7-59ed-4537-b4d1-31baf63a5466 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:13.230404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:07:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:13.234890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c2aa-595e-4f4b-9c22-1576f1796f91 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:13.235330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c2aa-595e-4f4b-9c22-1576f1796f91 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:13.235444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c2aa-595e-4f4b-9c22-1576f1796f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:13.303220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6dd3c2aa-595e-4f4b-9c22-1576f1796f91 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:13.304444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6dd3c2aa-595e-4f4b-9c22-1576f1796f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:13.322441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:13.401377 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:13.437561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53510a7-59ed-4537-b4d1-31baf63a5466 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:13.439117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e53510a7-59ed-4537-b4d1-31baf63a5466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:13.542511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53510a7-59ed-4537-b4d1-31baf63a5466 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:13.607954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:13.614783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5855afd-abce-4e47-8bf5-556c3ac1af02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:13.615391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5855afd-abce-4e47-8bf5-556c3ac1af02 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:13.615669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5855afd-abce-4e47-8bf5-556c3ac1af02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:13.615940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5855afd-abce-4e47-8bf5-556c3ac1af02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:13.617617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b5855afd-abce-4e47-8bf5-556c3ac1af02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:13.723722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5855afd-abce-4e47-8bf5-556c3ac1af02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:13.725172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5855afd-abce-4e47-8bf5-556c3ac1af02 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:13.742835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 742/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:15.745922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadd34b-7162-4f53-b5c4-7d6288c3b8d3 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:15.746468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadd34b-7162-4f53-b5c4-7d6288c3b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:15.746468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadd34b-7162-4f53-b5c4-7d6288c3b8d3 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:15.747795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:15.755269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4bdfe-bbe5-48bf-9b68-313482f1a109 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:15.755718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4bdfe-bbe5-48bf-9b68-313482f1a109 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:15.755815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4bdfe-bbe5-48bf-9b68-313482f1a109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:15.821627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2bc4bdfe-bbe5-48bf-9b68-313482f1a109 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:15.822836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2bc4bdfe-bbe5-48bf-9b68-313482f1a109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:15.871440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:15.919604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:15.956246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadd34b-7162-4f53-b5c4-7d6288c3b8d3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:15.957795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7aadd34b-7162-4f53-b5c4-7d6288c3b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:16.060903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadd34b-7162-4f53-b5c4-7d6288c3b8d3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:16.125169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:16.141109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f8c75-58a1-42fa-8f50-2b339539bf7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:16.141708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f8c75-58a1-42fa-8f50-2b339539bf7b service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:16.141983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f8c75-58a1-42fa-8f50-2b339539bf7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:16.142251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f8c75-58a1-42fa-8f50-2b339539bf7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:16.143872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-720f8c75-58a1-42fa-8f50-2b339539bf7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:16.253253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f8c75-58a1-42fa-8f50-2b339539bf7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:16.254679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f8c75-58a1-42fa-8f50-2b339539bf7b service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:16.272642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 727/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:25.917797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=22 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:26.270378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=66 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:35.800820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30cff5-75b3-418d-adb0-eb82ca03c53e None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:35.801393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30cff5-75b3-418d-adb0-eb82ca03c53e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:35.801393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30cff5-75b3-418d-adb0-eb82ca03c53e None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:35.802755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 744/1471] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:35.808079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f42d50-6e47-4431-864f-b67085373de7 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:35.808529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f42d50-6e47-4431-864f-b67085373de7 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:35.808714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f42d50-6e47-4431-864f-b67085373de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:35.878315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-36f42d50-6e47-4431-864f-b67085373de7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:35.879564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-36f42d50-6e47-4431-864f-b67085373de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:35.975951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 728/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:36.060908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e4319-ca85-4684-ba56-1190898db924 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:36.061191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e4319-ca85-4684-ba56-1190898db924 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:36.061540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e4319-ca85-4684-ba56-1190898db924 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:36.062926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 745/1473] 10.222.0.47 () {54 vars in 966 bytes} [Wed Jul 27 20:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:36.067928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37a6b0-926f-4cf2-bdef-baa24d33e4aa None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:36.068216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37a6b0-926f-4cf2-bdef-baa24d33e4aa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:36.068524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37a6b0-926f-4cf2-bdef-baa24d33e4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:36.143691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-1d37a6b0-926f-4cf2-bdef-baa24d33e4aa None None] MFA Rules not processed for user `813427e9e85a4b48a550a290211f39b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:36.144924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1d37a6b0-926f-4cf2-bdef-baa24d33e4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:36.165128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:36.219324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 729/1474] 10.222.0.47 () {58 vars in 1063 bytes} [Wed Jul 27 20:07:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:36.227532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e4319-ca85-4684-ba56-1190898db924 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:36.229116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a86e4319-ca85-4684-ba56-1190898db924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:36.335883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e4319-ca85-4684-ba56-1190898db924 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:36.410399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4e432-8d4c-402e-9684-a9bcf372990a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:36.411043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4e432-8d4c-402e-9684-a9bcf372990a service glance] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:36.411323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4e432-8d4c-402e-9684-a9bcf372990a service glance] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:36.411599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4e432-8d4c-402e-9684-a9bcf372990a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:36.413773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e4e4e432-8d4c-402e-9684-a9bcf372990a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:36.467630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:36.522530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4e432-8d4c-402e-9684-a9bcf372990a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:36.523991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4e432-8d4c-402e-9684-a9bcf372990a service glance] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:36.542458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 746/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:38.626676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e0eda-7923-4156-ab03-db3ac43b0abf None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:38.627092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e0eda-7923-4156-ab03-db3ac43b0abf None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:38.627322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e0eda-7923-4156-ab03-db3ac43b0abf None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:38.628631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 730/1476] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:38.634356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53b3c-7970-4f8a-a808-7df81abc90b4 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:38.634768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53b3c-7970-4f8a-a808-7df81abc90b4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:38.634908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a53b3c-7970-4f8a-a808-7df81abc90b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:38.700129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c4a53b3c-7970-4f8a-a808-7df81abc90b4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:38.701367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c4a53b3c-7970-4f8a-a808-7df81abc90b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:38.795485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 747/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:38.817815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e0eda-7923-4156-ab03-db3ac43b0abf None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:38.819826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9e7e0eda-7923-4156-ab03-db3ac43b0abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:38.942906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e0eda-7923-4156-ab03-db3ac43b0abf None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:38.992032 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:39.036709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f2ddb-c217-417d-975f-b07c571ed147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:39.037324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2ddb-c217-417d-975f-b07c571ed147 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:39.037752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2ddb-c217-417d-975f-b07c571ed147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:39.038026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f2ddb-c217-417d-975f-b07c571ed147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:39.038723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f2ddb-c217-417d-975f-b07c571ed147 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:39.040476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-9c8f2ddb-c217-417d-975f-b07c571ed147 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:07:39.040476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:07:39.040476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:07:39.041598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f2ddb-c217-417d-975f-b07c571ed147 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:39.060458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 731/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Jul 27 20:07:38 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:07:41.086553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46228b6c-a86b-4482-8c63-e8093174cea1 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:41.087109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46228b6c-a86b-4482-8c63-e8093174cea1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:41.087109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46228b6c-a86b-4482-8c63-e8093174cea1 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:41.088557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 748/1479] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:41.093802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c794d-7db2-47f8-a0f0-642e09fea836 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:41.094216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c794d-7db2-47f8-a0f0-642e09fea836 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:41.094389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c794d-7db2-47f8-a0f0-642e09fea836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:41.161064 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e44c794d-7db2-47f8-a0f0-642e09fea836 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:41.162297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e44c794d-7db2-47f8-a0f0-642e09fea836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:41.256850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 732/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:41.299126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46228b6c-a86b-4482-8c63-e8093174cea1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:41.300687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-46228b6c-a86b-4482-8c63-e8093174cea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:41.310913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:41.447518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46228b6c-a86b-4482-8c63-e8093174cea1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:41.525805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:41.559801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707ea1a-4f24-4b64-816c-adf1a2c3e253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:41.560450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ea1a-4f24-4b64-816c-adf1a2c3e253 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:41.560737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ea1a-4f24-4b64-816c-adf1a2c3e253 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:41.561012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ea1a-4f24-4b64-816c-adf1a2c3e253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:41.562676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f707ea1a-4f24-4b64-816c-adf1a2c3e253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:41.650915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707ea1a-4f24-4b64-816c-adf1a2c3e253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:41.652479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707ea1a-4f24-4b64-816c-adf1a2c3e253 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:41.669810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 749/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:43.839862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817fbbb-566a-4532-9ef3-89236a2f3b81 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:43.843341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817fbbb-566a-4532-9ef3-89236a2f3b81 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:43.843341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817fbbb-566a-4532-9ef3-89236a2f3b81 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:43.843341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 733/1482] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:43.847034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647a80-393e-4a04-91b9-daf933d1fd1c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:43.847463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647a80-393e-4a04-91b9-daf933d1fd1c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:43.847729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647a80-393e-4a04-91b9-daf933d1fd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:43.915093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-93647a80-393e-4a04-91b9-daf933d1fd1c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:43.916345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-93647a80-393e-4a04-91b9-daf933d1fd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:43.963801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:44.013895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 750/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:44.050948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817fbbb-566a-4532-9ef3-89236a2f3b81 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:44.052561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f817fbbb-566a-4532-9ef3-89236a2f3b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:44.078694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:44.159342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817fbbb-566a-4532-9ef3-89236a2f3b81 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:44.231442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13061b8-ddb9-4e2b-b03f-9d07af0f3aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:44.232054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13061b8-ddb9-4e2b-b03f-9d07af0f3aff service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:44.232334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13061b8-ddb9-4e2b-b03f-9d07af0f3aff service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:44.232644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13061b8-ddb9-4e2b-b03f-9d07af0f3aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:44.234257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a13061b8-ddb9-4e2b-b03f-9d07af0f3aff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:44.307103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:44.344551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13061b8-ddb9-4e2b-b03f-9d07af0f3aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:44.346025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13061b8-ddb9-4e2b-b03f-9d07af0f3aff service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:44.365703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 734/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:46.412079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9b4d4-ac43-4fe6-a420-51ed9a5c10dc None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:46.412676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9b4d4-ac43-4fe6-a420-51ed9a5c10dc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:46.412676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9b4d4-ac43-4fe6-a420-51ed9a5c10dc None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:46.414039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 751/1485] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:46.419307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66eda3e-c84f-4983-b7fd-f77d89b109ab None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:46.419737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66eda3e-c84f-4983-b7fd-f77d89b109ab None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:46.419849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66eda3e-c84f-4983-b7fd-f77d89b109ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:46.486609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f66eda3e-c84f-4983-b7fd-f77d89b109ab None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:46.487723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f66eda3e-c84f-4983-b7fd-f77d89b109ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:46.584866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 735/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:46.623773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9b4d4-ac43-4fe6-a420-51ed9a5c10dc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:46.625376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b0e9b4d4-ac43-4fe6-a420-51ed9a5c10dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:46.727871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9b4d4-ac43-4fe6-a420-51ed9a5c10dc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:46.799256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:46.801000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbfb0c-ae1b-4146-ae5b-c8edcf388c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:46.801621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfb0c-ae1b-4146-ae5b-c8edcf388c11 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:46.801905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfb0c-ae1b-4146-ae5b-c8edcf388c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:46.802179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbfb0c-ae1b-4146-ae5b-c8edcf388c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:46.803879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b6bbfb0c-ae1b-4146-ae5b-c8edcf388c11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:46.912077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbfb0c-ae1b-4146-ae5b-c8edcf388c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:46.913527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbfb0c-ae1b-4146-ae5b-c8edcf388c11 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:46.931456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 752/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:49.075167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10540-2062-4dff-af53-95da1ea34309 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:49.075720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10540-2062-4dff-af53-95da1ea34309 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:49.075720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10540-2062-4dff-af53-95da1ea34309 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:49.077089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 736/1488] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:49.082837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695208ed-862b-4c9f-ae42-b69a6c5d00ef None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:49.083539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695208ed-862b-4c9f-ae42-b69a6c5d00ef None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:49.084407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695208ed-862b-4c9f-ae42-b69a6c5d00ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:49.149931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-695208ed-862b-4c9f-ae42-b69a6c5d00ef None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:49.151146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-695208ed-862b-4c9f-ae42-b69a6c5d00ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:49.202340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:49.245704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 753/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:49.286639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea10540-2062-4dff-af53-95da1ea34309 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:49.288165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2ea10540-2062-4dff-af53-95da1ea34309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:49.323653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:49.392106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea10540-2062-4dff-af53-95da1ea34309 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:49.465777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef3693-ddf7-481c-9bb4-d75aa0b3fd4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:49.466398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef3693-ddf7-481c-9bb4-d75aa0b3fd4c service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:49.466680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef3693-ddf7-481c-9bb4-d75aa0b3fd4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:49.466950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef3693-ddf7-481c-9bb4-d75aa0b3fd4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:49.468683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e0ef3693-ddf7-481c-9bb4-d75aa0b3fd4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:49.548062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:49.577655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef3693-ddf7-481c-9bb4-d75aa0b3fd4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:49.579272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef3693-ddf7-481c-9bb4-d75aa0b3fd4c service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:49.597775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 737/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:07:52.017047 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18038d70-d228-4fc4-866b-6bd9ee95d9ac None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:52.017592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18038d70-d228-4fc4-866b-6bd9ee95d9ac None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:52.017592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18038d70-d228-4fc4-866b-6bd9ee95d9ac None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:52.018951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 754/1491] 10.222.0.47 () {54 vars in 926 bytes} [Wed Jul 27 20:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:07:52.024878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586afd9-61a1-4d34-9ca2-4b6916e5a7e0 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:52.025273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586afd9-61a1-4d34-9ca2-4b6916e5a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:52.025471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586afd9-61a1-4d34-9ca2-4b6916e5a7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:52.091993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a586afd9-61a1-4d34-9ca2-4b6916e5a7e0 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:07:52.093241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a586afd9-61a1-4d34-9ca2-4b6916e5a7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:52.193548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 738/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Wed Jul 27 20:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:07:52.232710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18038d70-d228-4fc4-866b-6bd9ee95d9ac None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:07:52.234256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-18038d70-d228-4fc4-866b-6bd9ee95d9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:52.336877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18038d70-d228-4fc4-866b-6bd9ee95d9ac None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:07:52.410243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022468a0-e94d-44bc-8b39-8a03f36292be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:07:52.410857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022468a0-e94d-44bc-8b39-8a03f36292be service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:07:52.411140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022468a0-e94d-44bc-8b39-8a03f36292be service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:07:52.411415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022468a0-e94d-44bc-8b39-8a03f36292be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:07:52.413107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-022468a0-e94d-44bc-8b39-8a03f36292be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:07:52.422540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:07:52.525032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022468a0-e94d-44bc-8b39-8a03f36292be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:07:52.526558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022468a0-e94d-44bc-8b39-8a03f36292be service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:07:52.545474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 755/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:02.191213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=82 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:02.543211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=50 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:29.573944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.574550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.574550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.651580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:29.652776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:29.747945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 739/1494] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:29.756054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40952e9f-6ed5-449b-9f4e-11702a7385fb None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.756562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40952e9f-6ed5-449b-9f4e-11702a7385fb None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.756666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40952e9f-6ed5-449b-9f4e-11702a7385fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.832996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-40952e9f-6ed5-449b-9f4e-11702a7385fb None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:29.834200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-40952e9f-6ed5-449b-9f4e-11702a7385fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:29.929355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 756/1495] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:29.934025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:29.935229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:29.947141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a96ad2-1bcc-4fc9-b0c7-230a5c8409ce None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.010051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:30.042276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764df6-babf-46db-890b-4778e793523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.043075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764df6-babf-46db-890b-4778e793523d admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.043417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764df6-babf-46db-890b-4778e793523d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.043887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35764df6-babf-46db-890b-4778e793523d admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.044924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35764df6-babf-46db-890b-4778e793523d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:09:30.045238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35764df6-babf-46db-890b-4778e793523d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:30.046256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-35764df6-babf-46db-890b-4778e793523d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:30.046256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:09:30.046256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:09:30.047142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35764df6-babf-46db-890b-4778e793523d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:30.058730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 740/1496] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:09:30.066907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40952e9f-6ed5-449b-9f4e-11702a7385fb None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.076609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40952e9f-6ed5-449b-9f4e-11702a7385fb None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.137290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:30.171090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.171706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.171989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.172262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.172962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:30.174770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING py.warnings [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:30.174770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: warnings.warn(msg) Jul 27 20:09:30.174770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: Jul 27 20:09:30.175949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:30.295756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 757/1497] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:30 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:30.299994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764df6-babf-46db-890b-4778e793523d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.301534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-35764df6-babf-46db-890b-4778e793523d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.334627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35764df6-babf-46db-890b-4778e793523d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.426526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.427131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.427404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.427671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.428331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:30.429957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:30.429957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:09:30.429957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:09:30.431141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:30.555246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 741/1498] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:30 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:30.559864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.561540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.624491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e428a-9f91-4370-ab1e-6cf17d386bbb admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.707098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:30.717712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.718330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.718610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.718884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.719578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:30.720627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING py.warnings [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:30.720627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: warnings.warn(msg) Jul 27 20:09:30.720627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: Jul 27 20:09:30.721510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:30.734283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 758/1499] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:30.738273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.740157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.751725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283600e8-c206-44d1-91f8-fa29287c0812 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.778416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:30.850078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a303f4-8621-4841-81ee-8a4d226e0d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.851232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a303f4-8621-4841-81ee-8a4d226e0d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.851729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a303f4-8621-4841-81ee-8a4d226e0d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.852177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a303f4-8621-4841-81ee-8a4d226e0d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.853339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a303f4-8621-4841-81ee-8a4d226e0d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:30.854359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-75a303f4-8621-4841-81ee-8a4d226e0d95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:30.854359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:09:30.854359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:09:30.855244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a303f4-8621-4841-81ee-8a4d226e0d95 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:30.867728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 742/1500] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:30.871839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.873387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.908076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3261e6-c61e-4105-8cca-04fccfb9c00d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:31.001166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110168e-920e-45e4-9f66-531095840f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:31.001830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110168e-920e-45e4-9f66-531095840f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.002112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110168e-920e-45e4-9f66-531095840f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.002388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110168e-920e-45e4-9f66-531095840f18 admin admin] PATH_INFO: `/v3/projects/61d9f74a1a724c20a57c43d1545f0879/users/7fa0106d13844342840d70b194f4af53/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.025577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110168e-920e-45e4-9f66-531095840f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d9f74a1a724c20a57c43d1545f0879, user_id=7fa0106d13844342840d70b194f4af53, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:31.026639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING py.warnings [None req-d110168e-920e-45e4-9f66-531095840f18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:31.026639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: warnings.warn(msg) Jul 27 20:09:31.026639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: Jul 27 20:09:31.028617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110168e-920e-45e4-9f66-531095840f18 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:31.114002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 759/1501] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:30 2022] PUT /identity/v3/projects/61d9f74a1a724c20a57c43d1545f0879/users/7fa0106d13844342840d70b194f4af53/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:31.126186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38e9d5-655b-4e0c-bf1f-d35914cf3c4c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.127188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38e9d5-655b-4e0c-bf1f-d35914cf3c4c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.127188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38e9d5-655b-4e0c-bf1f-d35914cf3c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.183473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-cb38e9d5-655b-4e0c-bf1f-d35914cf3c4c None None] MFA Rules not processed for user `7fa0106d13844342840d70b194f4af53`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:31.184695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cb38e9d5-655b-4e0c-bf1f-d35914cf3c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:31.260137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 743/1502] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:31.273725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f4d7-0881-4fc1-b823-b9c8e27e7f48 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.274010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f4d7-0881-4fc1-b823-b9c8e27e7f48 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.274285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2f4d7-0881-4fc1-b823-b9c8e27e7f48 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.275643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 760/1503] 10.222.0.47 () {54 vars in 964 bytes} [Wed Jul 27 20:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:09:31.280583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14565c-7674-4670-be64-4a47f8e8f6f2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.280873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14565c-7674-4670-be64-4a47f8e8f6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.281150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14565c-7674-4670-be64-4a47f8e8f6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.312050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:31.355676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5f14565c-7674-4670-be64-4a47f8e8f6f2 None None] MFA Rules not processed for user `78776208cde94134b478ceee6cc70ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:31.356931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5f14565c-7674-4670-be64-4a47f8e8f6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:31.433131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 744/1504] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Jul 27 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:31.440734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2f4d7-0881-4fc1-b823-b9c8e27e7f48 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:31.442239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bcb2f4d7-0881-4fc1-b823-b9c8e27e7f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:31.491006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:31.545915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2f4d7-0881-4fc1-b823-b9c8e27e7f48 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:31.621334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:31.621952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.622233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.622507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.624141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:31.720443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:31.721808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:31.725297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:31.739881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 761/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:31.784538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f7a37-6e6c-4c3a-874c-8a82d16fc88b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.784943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f7a37-6e6c-4c3a-874c-8a82d16fc88b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.785087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f7a37-6e6c-4c3a-874c-8a82d16fc88b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.786339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 745/1506] 10.222.0.47 () {52 vars in 830 bytes} [Wed Jul 27 20:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:09:31.790355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:31.791624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:31.895646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98754d1-9e74-4c4e-88d6-4f4645b50495 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:31.992249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:31.992918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.993196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.993467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] PATH_INFO: `/v3/users/7fa0106d13844342840d70b194f4af53` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.005787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa0106d13844342840d70b194f4af53)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:32.006814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING py.warnings [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:32.006814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: warnings.warn(msg) Jul 27 20:09:32.006814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: Jul 27 20:09:32.008013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.017532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:32.222795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa0106d13844342840d70b194f4af53'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.223332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa0106d13844342840d70b194f4af53'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.285231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa0106d13844342840d70b194f4af53'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.303768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa0106d13844342840d70b194f4af53'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.315958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa0106d13844342840d70b194f4af53'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.321623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 762/1507] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:31 2022] DELETE /identity/v3/users/7fa0106d13844342840d70b194f4af53 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:32.325960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.326328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.326710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.383984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:32.385243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:32.461436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:32.473604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 746/1508] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:32.483692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:32.485118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:32.584562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25c5b8-3f19-4c20-9cd5-5f872dc9a80a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:32.642787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:32.663164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd44af-5b07-420d-8b3a-0ac5171deb17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:32.663882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd44af-5b07-420d-8b3a-0ac5171deb17 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.664173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd44af-5b07-420d-8b3a-0ac5171deb17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.664494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd44af-5b07-420d-8b3a-0ac5171deb17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.666127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-11fd44af-5b07-420d-8b3a-0ac5171deb17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:32.781506 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd44af-5b07-420d-8b3a-0ac5171deb17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:32.782966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd44af-5b07-420d-8b3a-0ac5171deb17 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.800899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 763/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:33.304446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:33.306316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:33.318181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850bf52-6d8b-4b79-9426-e6a9991963df None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:33.418682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:33.419252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:33.419558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:33.419835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] PATH_INFO: `/v3/projects/61d9f74a1a724c20a57c43d1545f0879` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:33.429628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d9f74a1a724c20a57c43d1545f0879)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:33.431279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:09:33.431279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:09:33.431279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:09:33.432495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:33.570077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d9f74a1a724c20a57c43d1545f0879', 'request_id': 'req-8613ccca-8948-417e-aa1e-bd1cc4f86466'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:33.582987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d9f74a1a724c20a57c43d1545f0879', 'request_id': 'req-8613ccca-8948-417e-aa1e-bd1cc4f86466'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:33.589909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d9f74a1a724c20a57c43d1545f0879', 'request_id': 'req-8613ccca-8948-417e-aa1e-bd1cc4f86466'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:33.668669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] The token cache is being invalidate because project 61d9f74a1a724c20a57c43d1545f0879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:33.669013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:33.669493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-8613ccca-8948-417e-aa1e-bd1cc4f86466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:33.670896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 747/1510] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:33 2022] DELETE /identity/v3/projects/61d9f74a1a724c20a57c43d1545f0879 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:43.006157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=62 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:43.666181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:13.781975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73d039-2395-4466-a481-51f39c26122e None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:13.783913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73d039-2395-4466-a481-51f39c26122e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:13.783913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73d039-2395-4466-a481-51f39c26122e None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:13.784214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:11:13.788625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cb8f-0782-4bbc-923f-5e077905fbd1 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:13.789034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cb8f-0782-4bbc-923f-5e077905fbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:13.789174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1cb8f-0782-4bbc-923f-5e077905fbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:13.870679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-49b1cb8f-0782-4bbc-923f-5e077905fbd1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:13.871896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-49b1cb8f-0782-4bbc-923f-5e077905fbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:13.983523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:14.136002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73d039-2395-4466-a481-51f39c26122e None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:14.138182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cc73d039-2395-4466-a481-51f39c26122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:14.261837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73d039-2395-4466-a481-51f39c26122e None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:14.267322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d04f7-fd4b-49d5-9076-643376ac3192 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:14.267936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d04f7-fd4b-49d5-9076-643376ac3192 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:14.268221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d04f7-fd4b-49d5-9076-643376ac3192 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:14.268530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d04f7-fd4b-49d5-9076-643376ac3192 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:14.277072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d04f7-fd4b-49d5-9076-643376ac3192 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:14.278643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d04f7-fd4b-49d5-9076-643376ac3192 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:14.287970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 765/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:15.104678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db250e2-c73c-4c69-bc55-50ba91df3509 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:15.105181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db250e2-c73c-4c69-bc55-50ba91df3509 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:15.105181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db250e2-c73c-4c69-bc55-50ba91df3509 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:15.106530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:11:15.111344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d36735-b0ea-41e7-9cf7-cc263095d7ff None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:15.111885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d36735-b0ea-41e7-9cf7-cc263095d7ff None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:15.111885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d36735-b0ea-41e7-9cf7-cc263095d7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:15.131201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-17d36735-b0ea-41e7-9cf7-cc263095d7ff None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:15.132405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-17d36735-b0ea-41e7-9cf7-cc263095d7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:15.141005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:15.292355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db250e2-c73c-4c69-bc55-50ba91df3509 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:15.307447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db250e2-c73c-4c69-bc55-50ba91df3509 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:15.312960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9c9e7-45eb-4fd1-9c9f-a98b2cfd9688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:15.313568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9c9e7-45eb-4fd1-9c9f-a98b2cfd9688 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:15.313853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9c9e7-45eb-4fd1-9c9f-a98b2cfd9688 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:15.314129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9c9e7-45eb-4fd1-9c9f-a98b2cfd9688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:15.322209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9c9e7-45eb-4fd1-9c9f-a98b2cfd9688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:15.323797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9c9e7-45eb-4fd1-9c9f-a98b2cfd9688 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:15.335372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 750/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:15.411838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066a13-2435-4548-9ea5-95d82da8d222 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:15.412128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066a13-2435-4548-9ea5-95d82da8d222 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:15.412488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066a13-2435-4548-9ea5-95d82da8d222 None None] PATH_INFO: `/v3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:15.413831 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 767/1517] 10.222.0.69 () {54 vars in 925 bytes} [Wed Jul 27 20:11:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:11:15.447692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ede4d8-51da-4b45-a643-9a421d1ac529 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:15.447975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ede4d8-51da-4b45-a643-9a421d1ac529 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:15.448253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ede4d8-51da-4b45-a643-9a421d1ac529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:15.508838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-41ede4d8-51da-4b45-a643-9a421d1ac529 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:15.510061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-41ede4d8-51da-4b45-a643-9a421d1ac529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:15.526942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:15.579959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 751/1518] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Jul 27 20:11:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:24.900799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484710ed-f236-4789-812a-ecd3803c2a0b None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:24.901353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484710ed-f236-4789-812a-ecd3803c2a0b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:24.901353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484710ed-f236-4789-812a-ecd3803c2a0b None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:24.902644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 768/1519] 10.222.0.47 () {54 vars in 961 bytes} [Wed Jul 27 20:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:11:24.907735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9f77a-6c7a-4cec-81f5-0ff6a340afa1 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:24.908172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9f77a-6c7a-4cec-81f5-0ff6a340afa1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:24.908277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9f77a-6c7a-4cec-81f5-0ff6a340afa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:24.927986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d6a9f77a-6c7a-4cec-81f5-0ff6a340afa1 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:24.929212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d6a9f77a-6c7a-4cec-81f5-0ff6a340afa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:24.938568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 752/1520] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Jul 27 20:11:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:24.945941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484710ed-f236-4789-812a-ecd3803c2a0b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:24.959016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484710ed-f236-4789-812a-ecd3803c2a0b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:24.963862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930d7ff-8a0d-4690-994e-7f655d2c977b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:24.964480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930d7ff-8a0d-4690-994e-7f655d2c977b service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:24.964757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930d7ff-8a0d-4690-994e-7f655d2c977b service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:24.965025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930d7ff-8a0d-4690-994e-7f655d2c977b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:24.973115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930d7ff-8a0d-4690-994e-7f655d2c977b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:24.974665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930d7ff-8a0d-4690-994e-7f655d2c977b service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:24.984828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 769/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:11:26.568786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba507e7c-e0d9-4970-ae26-62b46bd99184 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:26.569325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba507e7c-e0d9-4970-ae26-62b46bd99184 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:26.569325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba507e7c-e0d9-4970-ae26-62b46bd99184 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:26.570688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:11:26.575554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a598-f472-4645-9d70-fb13c0e3640b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:26.575977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a598-f472-4645-9d70-fb13c0e3640b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:26.576137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078a598-f472-4645-9d70-fb13c0e3640b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:26.595178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b078a598-f472-4645-9d70-fb13c0e3640b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:26.596313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b078a598-f472-4645-9d70-fb13c0e3640b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:26.604736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:26.757072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba507e7c-e0d9-4970-ae26-62b46bd99184 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:26.771415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba507e7c-e0d9-4970-ae26-62b46bd99184 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:26.778531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c739179-8e0b-49ae-825c-2a6a37a4c760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:26.778531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739179-8e0b-49ae-825c-2a6a37a4c760 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:26.778531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739179-8e0b-49ae-825c-2a6a37a4c760 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:26.778531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739179-8e0b-49ae-825c-2a6a37a4c760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:26.786338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c739179-8e0b-49ae-825c-2a6a37a4c760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:26.787891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c739179-8e0b-49ae-825c-2a6a37a4c760 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:26.799948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 754/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:27.619996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89532f9-6237-4dbe-ae9a-ec89aa5615d1 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:27.620531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89532f9-6237-4dbe-ae9a-ec89aa5615d1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:27.620531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89532f9-6237-4dbe-ae9a-ec89aa5615d1 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:27.621895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:11:27.626836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba000d-0199-48d1-83e4-88839f63b419 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:27.627243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba000d-0199-48d1-83e4-88839f63b419 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:27.627394 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba000d-0199-48d1-83e4-88839f63b419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:27.647349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-9dba000d-0199-48d1-83e4-88839f63b419 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:27.648578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9dba000d-0199-48d1-83e4-88839f63b419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:27.656774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:27.809985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89532f9-6237-4dbe-ae9a-ec89aa5615d1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:27.824999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89532f9-6237-4dbe-ae9a-ec89aa5615d1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:27.830412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88931952-57c0-4fcf-a1e7-a9d993f2f390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:27.831014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931952-57c0-4fcf-a1e7-a9d993f2f390 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:27.831295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931952-57c0-4fcf-a1e7-a9d993f2f390 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:27.831571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931952-57c0-4fcf-a1e7-a9d993f2f390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:27.839632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88931952-57c0-4fcf-a1e7-a9d993f2f390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:27.841249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88931952-57c0-4fcf-a1e7-a9d993f2f390 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:27.850584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 772/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:31.641895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2aa9f4-428b-4efd-8a18-08f4b5c71f73 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:31.642464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2aa9f4-428b-4efd-8a18-08f4b5c71f73 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:31.642464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2aa9f4-428b-4efd-8a18-08f4b5c71f73 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:31.643758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:11:31.648886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a2184-3d6b-46ea-8b7f-45e8be7d4f3f None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:31.649293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a2184-3d6b-46ea-8b7f-45e8be7d4f3f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:31.649442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a2184-3d6b-46ea-8b7f-45e8be7d4f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:31.669062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c30a2184-3d6b-46ea-8b7f-45e8be7d4f3f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:31.670256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c30a2184-3d6b-46ea-8b7f-45e8be7d4f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:31.678037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:31.750179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2aa9f4-428b-4efd-8a18-08f4b5c71f73 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:31.751914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6b2aa9f4-428b-4efd-8a18-08f4b5c71f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:31.829823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2aa9f4-428b-4efd-8a18-08f4b5c71f73 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:31.833445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022999f-e89a-492f-8349-210f2794f27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:31.834071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022999f-e89a-492f-8349-210f2794f27d service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:31.834351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022999f-e89a-492f-8349-210f2794f27d service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:31.834626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022999f-e89a-492f-8349-210f2794f27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:31.844532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022999f-e89a-492f-8349-210f2794f27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:31.845960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022999f-e89a-492f-8349-210f2794f27d service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:31.855577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 757/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:33.849572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f433f3-af85-486d-b685-ad1d082695d2 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:33.850132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f433f3-af85-486d-b685-ad1d082695d2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:33.850132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f433f3-af85-486d-b685-ad1d082695d2 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:33.851469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:11:33.857287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1fff5-25e5-4313-a451-6a8bb0af3c2c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:33.857681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1fff5-25e5-4313-a451-6a8bb0af3c2c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:33.857851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1fff5-25e5-4313-a451-6a8bb0af3c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:33.878022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-39e1fff5-25e5-4313-a451-6a8bb0af3c2c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:33.879219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-39e1fff5-25e5-4313-a451-6a8bb0af3c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:33.887090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:33.922624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f433f3-af85-486d-b685-ad1d082695d2 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:33.934265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f433f3-af85-486d-b685-ad1d082695d2 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:33.938000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e1be-3c37-4efb-bdba-d3a4880032df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:33.944870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e1be-3c37-4efb-bdba-d3a4880032df service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:33.944870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e1be-3c37-4efb-bdba-d3a4880032df service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:33.944870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e1be-3c37-4efb-bdba-d3a4880032df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:33.948112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e1be-3c37-4efb-bdba-d3a4880032df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:33.949563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e1be-3c37-4efb-bdba-d3a4880032df service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:33.959019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 775/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:37.536086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9c1a-84af-4f8a-a85c-1162131e138c None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:37.536826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9c1a-84af-4f8a-a85c-1162131e138c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:37.536826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e9c1a-84af-4f8a-a85c-1162131e138c None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:37.538016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:11:37.543735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974e7e-17aa-4233-a010-83742d91beb4 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:37.544137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974e7e-17aa-4233-a010-83742d91beb4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:37.544522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7974e7e-17aa-4233-a010-83742d91beb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:37.563576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a7974e7e-17aa-4233-a010-83742d91beb4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:37.564816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a7974e7e-17aa-4233-a010-83742d91beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:37.573106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:37.876444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9c1a-84af-4f8a-a85c-1162131e138c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:37.878136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-100e9c1a-84af-4f8a-a85c-1162131e138c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:37.941921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:37.955300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e9c1a-84af-4f8a-a85c-1162131e138c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:37.958906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769bcc8-ad69-42d7-9c57-d1437aeb5d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:37.965084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769bcc8-ad69-42d7-9c57-d1437aeb5d52 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:37.965084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769bcc8-ad69-42d7-9c57-d1437aeb5d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:37.965084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769bcc8-ad69-42d7-9c57-d1437aeb5d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:37.969753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769bcc8-ad69-42d7-9c57-d1437aeb5d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:37.971180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769bcc8-ad69-42d7-9c57-d1437aeb5d52 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:37.981067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 760/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:39.995235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4398a-b7f0-4e48-9052-479dfe13a71a None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:39.995846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4398a-b7f0-4e48-9052-479dfe13a71a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:39.995846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4398a-b7f0-4e48-9052-479dfe13a71a None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:39.997268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:11:40.002696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381bd46-88dc-4fe1-a8a7-5cf1a34890ac None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:40.003139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381bd46-88dc-4fe1-a8a7-5cf1a34890ac None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:40.003245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381bd46-88dc-4fe1-a8a7-5cf1a34890ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:40.022984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d381bd46-88dc-4fe1-a8a7-5cf1a34890ac None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:40.024183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d381bd46-88dc-4fe1-a8a7-5cf1a34890ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:40.033268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:40.334778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4398a-b7f0-4e48-9052-479dfe13a71a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:40.336495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-46f4398a-b7f0-4e48-9052-479dfe13a71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:40.349406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4398a-b7f0-4e48-9052-479dfe13a71a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:40.352934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40129dc-e725-4fc1-ae8c-35adc306e2f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:40.353536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40129dc-e725-4fc1-ae8c-35adc306e2f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:40.353823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40129dc-e725-4fc1-ae8c-35adc306e2f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:40.354098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40129dc-e725-4fc1-ae8c-35adc306e2f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:40.362959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40129dc-e725-4fc1-ae8c-35adc306e2f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:40.364429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40129dc-e725-4fc1-ae8c-35adc306e2f3 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:40.374696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 778/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:50.018247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=11 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:50.370444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:09.913158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a31e8-189c-4d54-8501-79607a25f9bd None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:09.917200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a31e8-189c-4d54-8501-79607a25f9bd None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:09.917200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a31e8-189c-4d54-8501-79607a25f9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:09.940744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-569a31e8-189c-4d54-8501-79607a25f9bd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:09.942322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-569a31e8-189c-4d54-8501-79607a25f9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:09.955675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 762/1540] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:09.964426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82e79-7314-4d77-8064-7f0b88ec2b7c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:09.965005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82e79-7314-4d77-8064-7f0b88ec2b7c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:09.965005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc82e79-7314-4d77-8064-7f0b88ec2b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:09.999834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-8dc82e79-7314-4d77-8064-7f0b88ec2b7c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.001148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8dc82e79-7314-4d77-8064-7f0b88ec2b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.014929 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 779/1541] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.019639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a31e8-189c-4d54-8501-79607a25f9bd None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.034218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a31e8-189c-4d54-8501-79607a25f9bd None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.043189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6130c4c-3883-4f14-a5de-142ef7e2d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.043837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130c4c-3883-4f14-a5de-142ef7e2d356 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.044116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130c4c-3883-4f14-a5de-142ef7e2d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.044402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6130c4c-3883-4f14-a5de-142ef7e2d356 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.045113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6130c4c-3883-4f14-a5de-142ef7e2d356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:10.045416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6130c4c-3883-4f14-a5de-142ef7e2d356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.047090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6130c4c-3883-4f14-a5de-142ef7e2d356 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.068870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 763/1542] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:10.077329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc82e79-7314-4d77-8064-7f0b88ec2b7c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.094364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc82e79-7314-4d77-8064-7f0b88ec2b7c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.097601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b931b1-c620-47da-97d9-51c0168128f7 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.102120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b931b1-c620-47da-97d9-51c0168128f7 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.102120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b931b1-c620-47da-97d9-51c0168128f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.104467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b7d5a-d7fe-45b8-8e57-9dd728c02d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.105103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7d5a-d7fe-45b8-8e57-9dd728c02d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.105388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7d5a-d7fe-45b8-8e57-9dd728c02d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.105664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7d5a-d7fe-45b8-8e57-9dd728c02d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.106341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b7d5a-d7fe-45b8-8e57-9dd728c02d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.108239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b7d5a-d7fe-45b8-8e57-9dd728c02d17 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.133448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-10b931b1-c620-47da-97d9-51c0168128f7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.134966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-10b931b1-c620-47da-97d9-51c0168128f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.154426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 764/1543] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.155727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c403d-5482-4d53-a8e6-5f56b4b1cf8f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.156341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c403d-5482-4d53-a8e6-5f56b4b1cf8f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.156341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c403d-5482-4d53-a8e6-5f56b4b1cf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.184687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-be8c403d-5482-4d53-a8e6-5f56b4b1cf8f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.186538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-be8c403d-5482-4d53-a8e6-5f56b4b1cf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.201673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 765/1544] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.203110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d5c2-c407-446f-bf5c-c5b64361e361 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.203402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d5c2-c407-446f-bf5c-c5b64361e361 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.203678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d5c2-c407-446f-bf5c-c5b64361e361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.226186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-cd00d5c2-c407-446f-bf5c-c5b64361e361 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.227937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cd00d5c2-c407-446f-bf5c-c5b64361e361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.237818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 766/1545] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.243037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568fae8-adae-4ca5-be0c-dd544d79907f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.243336 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568fae8-adae-4ca5-be0c-dd544d79907f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.243609 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568fae8-adae-4ca5-be0c-dd544d79907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.267100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6568fae8-adae-4ca5-be0c-dd544d79907f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.268586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6568fae8-adae-4ca5-be0c-dd544d79907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.277803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 780/1546] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:10.277803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298377e-9415-4404-bc17-ad16390cd80c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.277803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298377e-9415-4404-bc17-ad16390cd80c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.277803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298377e-9415-4404-bc17-ad16390cd80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.322322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4298377e-9415-4404-bc17-ad16390cd80c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.325358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4298377e-9415-4404-bc17-ad16390cd80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.376415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 767/1547] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.382135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2387b91-81f4-4cbd-94c6-2a476e9c3912 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.382426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2387b91-81f4-4cbd-94c6-2a476e9c3912 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.384420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2387b91-81f4-4cbd-94c6-2a476e9c3912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.405141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c2387b91-81f4-4cbd-94c6-2a476e9c3912 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.406654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c2387b91-81f4-4cbd-94c6-2a476e9c3912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.412025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 781/1548] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.412957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298377e-9415-4404-bc17-ad16390cd80c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.415316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 768/1549] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.415976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2387b91-81f4-4cbd-94c6-2a476e9c3912 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.425435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2387b91-81f4-4cbd-94c6-2a476e9c3912 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.425622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298377e-9415-4404-bc17-ad16390cd80c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.429984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb07de-6a61-46b7-8c7f-2d50e8408942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.430240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.430582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb07de-6a61-46b7-8c7f-2d50e8408942 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.430786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.430914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb07de-6a61-46b7-8c7f-2d50e8408942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.431074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.431201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb07de-6a61-46b7-8c7f-2d50e8408942 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.431353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.432533 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb07de-6a61-46b7-8c7f-2d50e8408942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.432666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:10.432973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.433985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING py.warnings [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:13:10.433985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: warnings.warn(msg) Jul 27 20:13:10.433985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: Jul 27 20:13:10.434437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb07de-6a61-46b7-8c7f-2d50e8408942 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.434903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fdd82-96f0-40cd-a144-531916c4c91c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.446726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 782/1550] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:10.448714 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd8706-1d9a-4aee-99da-3d09053f315f None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.449010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd8706-1d9a-4aee-99da-3d09053f315f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.449290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd8706-1d9a-4aee-99da-3d09053f315f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.469025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-e1cd8706-1d9a-4aee-99da-3d09053f315f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.470220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e1cd8706-1d9a-4aee-99da-3d09053f315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.478289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 783/1551] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.479420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5fd6-dd4a-4fb8-8ad1-a0b7bf1acce3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.479704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5fd6-dd4a-4fb8-8ad1-a0b7bf1acce3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.479981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc5fd6-dd4a-4fb8-8ad1-a0b7bf1acce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.499634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-dafc5fd6-dd4a-4fb8-8ad1-a0b7bf1acce3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.500950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dafc5fd6-dd4a-4fb8-8ad1-a0b7bf1acce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.508852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 784/1552] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.510585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f8a0d-c919-48f0-ad6e-f0a00dacf663 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.510871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f8a0d-c919-48f0-ad6e-f0a00dacf663 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.511148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f8a0d-c919-48f0-ad6e-f0a00dacf663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.527054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 769/1553] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:10.528660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904a199-cab8-4b2a-b69c-f456da1aa48e None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.528950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904a199-cab8-4b2a-b69c-f456da1aa48e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.529228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904a199-cab8-4b2a-b69c-f456da1aa48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.531348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d72f8a0d-c919-48f0-ad6e-f0a00dacf663 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.532606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d72f8a0d-c919-48f0-ad6e-f0a00dacf663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.540743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 785/1554] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.542467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318c9f2-a234-4d2a-b2f9-de84e526191e None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.542755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318c9f2-a234-4d2a-b2f9-de84e526191e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.543044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318c9f2-a234-4d2a-b2f9-de84e526191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.549527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8904a199-cab8-4b2a-b69c-f456da1aa48e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.550723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8904a199-cab8-4b2a-b69c-f456da1aa48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.559546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 770/1555] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.560325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904a199-cab8-4b2a-b69c-f456da1aa48e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.565413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9318c9f2-a234-4d2a-b2f9-de84e526191e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.566667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9318c9f2-a234-4d2a-b2f9-de84e526191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.571928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904a199-cab8-4b2a-b69c-f456da1aa48e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.574722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 786/1556] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.575543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318c9f2-a234-4d2a-b2f9-de84e526191e None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.578846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd6ecb-eed7-4bce-b7dc-fdcbefd2cd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.579471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd6ecb-eed7-4bce-b7dc-fdcbefd2cd20 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.579783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd6ecb-eed7-4bce-b7dc-fdcbefd2cd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.580276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd6ecb-eed7-4bce-b7dc-fdcbefd2cd20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.581049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd6ecb-eed7-4bce-b7dc-fdcbefd2cd20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:10.581379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd6ecb-eed7-4bce-b7dc-fdcbefd2cd20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.583053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd6ecb-eed7-4bce-b7dc-fdcbefd2cd20 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.584766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318c9f2-a234-4d2a-b2f9-de84e526191e None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.589482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029e922-128f-42dc-9023-562e3ede2f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.590118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029e922-128f-42dc-9023-562e3ede2f54 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.590405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029e922-128f-42dc-9023-562e3ede2f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.590685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029e922-128f-42dc-9023-562e3ede2f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.591350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029e922-128f-42dc-9023-562e3ede2f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.593269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029e922-128f-42dc-9023-562e3ede2f54 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.601688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 771/1557] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:10.601688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b75a03-081d-4fea-aa57-0e1a39ad6895 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.601688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b75a03-081d-4fea-aa57-0e1a39ad6895 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.601688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b75a03-081d-4fea-aa57-0e1a39ad6895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.615845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f4b75a03-081d-4fea-aa57-0e1a39ad6895 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.617110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f4b75a03-081d-4fea-aa57-0e1a39ad6895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.625851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 772/1558] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.627367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490338a-c580-423e-8665-28464f5b6f7e None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.627367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490338a-c580-423e-8665-28464f5b6f7e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.627610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490338a-c580-423e-8665-28464f5b6f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.645916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-2490338a-c580-423e-8665-28464f5b6f7e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.647078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2490338a-c580-423e-8665-28464f5b6f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.654953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 773/1559] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.655858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490338a-c580-423e-8665-28464f5b6f7e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.664795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490338a-c580-423e-8665-28464f5b6f7e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.669833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.670399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.670681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.670955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.671628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.673293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.683367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 774/1560] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:10.686292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.693421 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce822b87-0b3c-494c-b78a-1005cc711ec2 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.698472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.699021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.699299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.699569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.700226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:10.700559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.702067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.710186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 787/1561] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:10.711519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd2138-5c79-4433-ab32-5a84a668a6f9 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.711805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd2138-5c79-4433-ab32-5a84a668a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.712078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd2138-5c79-4433-ab32-5a84a668a6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.716617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 775/1562] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:10.717948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.728787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06b58a-edc9-481b-8a1b-7ab8d0c71af6 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.732850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4cdd2138-5c79-4433-ab32-5a84a668a6f9 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:10.734056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4cdd2138-5c79-4433-ab32-5a84a668a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:10.768681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:10.786729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.787468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.787751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.788027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.788748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:10.789058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.790802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.792439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 788/1563] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:10.793338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd2138-5c79-4433-ab32-5a84a668a6f9 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.798123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:10.809944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 776/1564] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:10.809944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.811503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd2138-5c79-4433-ab32-5a84a668a6f9 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.811503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.811503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.811503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.812043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.812043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.813226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.814558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ec7bb-fefe-4713-9552-f3754e67fe88 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.819348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.819958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.820239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.820551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.821274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:10.821584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.824033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.836375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 777/1565] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:10.837195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.847148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7a221-25da-4a31-b026-a563f49894db admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.851923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.852548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.852831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.853105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.853851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:10.854153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.855717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.867140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 778/1566] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:10.868063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.878212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c3ac4-a49c-4421-b0a5-c28c088d657e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.882770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.883330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.883611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.883889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.884657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.886768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.893548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 789/1567] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:10.894360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.900588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 779/1568] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:10.904206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:10.914148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4e6b4-b240-486b-9ed9-6006d4750a14 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.956271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb975b-c299-43c1-b35c-7d496c494ef1 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:10.961501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.962122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.962404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.962685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.963379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.964471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING py.warnings [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:13:10.964471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: warnings.warn(msg) Jul 27 20:13:10.964471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: Jul 27 20:13:10.965676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:10.971587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:10.972201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:10.972511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:10.972787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:10.973450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:10.974462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:13:10.974462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:13:10.974462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:13:10.975770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.084501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 790/1569] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/users => generated 303 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.085339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.095062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371e1e2-90c3-42ee-bd77-8bee61128b12 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.099764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.100369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.100681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.100960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.101656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:11.101964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.103553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.118132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 780/1570] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:10 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.118132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.129173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 791/1571] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:11.129969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.140482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cc08e-40dc-401f-a5f7-726c0a1e0bff admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.184299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c89177-d861-4a42-9fbc-a0db96175fb7 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.189511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.190294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.190577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.190926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.191550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.193556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.197958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.198629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.198898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.199177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.199848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.201775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.315801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 781/1572] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.316831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.327011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:11.350425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 792/1573] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.356561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.388803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89298d7-94b2-452c-9111-6c75ce673337 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.396554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:11.414283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe9d42-1a39-46ea-abb6-b2481265dd14 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.419613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.420277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.420596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.420902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.421735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.424011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.446796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.447417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.447698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.447971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.448674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.450549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.507083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 793/1574] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.507919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.517535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f25aa2-2b63-49ca-bb12-240fac3d3c6f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.522647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.522825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.523109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.523384 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] PATH_INFO: `/v3/projects/480e9449f5e9458eb64fa91701d5be6c/users/6f49507b26104b318153db0cdf993740/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.539137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480e9449f5e9458eb64fa91701d5be6c, user_id=6f49507b26104b318153db0cdf993740, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.542438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.549587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 782/1575] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/projects => generated 405 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.551386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.609413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d27bd-80be-4324-bfed-ff599e73f19c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.613859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.614455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.614731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.615109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.615856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.617463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.628167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 783/1576] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:11.628957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.636257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 794/1577] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:11 2022] PUT /identity/v3/projects/480e9449f5e9458eb64fa91701d5be6c/users/6f49507b26104b318153db0cdf993740/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:11.637026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.638631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24afb1-5a2a-4c0c-90b1-cba1510e6308 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.677032 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:11.701278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.701896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.702181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.702460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.703129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.703304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a50f81-378c-4927-b572-9915db736344 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.705117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.707945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.708593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.708877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.709150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.709819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.711758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.833984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 784/1578] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.834794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.835112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 795/1579] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:11.838675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:11.852418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcae198-2dd1-44f9-953e-88a9522d7da3 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.906334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aaf69-ce8b-4131-8d25-de558d8df55a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:11.910859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.911469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.911747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.912016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.912702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.912931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:11.913556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:11.913838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:11.914118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:11.914734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:11.914844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:11.916772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.028941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 796/1580] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:12.029766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.039010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eac2fb-f699-477e-bca9-3cfdde18bf7c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.044186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.044793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.045073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.045343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.046044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.047621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.058821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 797/1581] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.059559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.068697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5662eb-5162-4799-aa6e-e632b97f940f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.073305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.073856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.074130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.074402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.075032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.076996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.087479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 785/1582] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:11 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:12.087479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.087782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:12.095831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714899d-77d9-4b4d-b6fc-a42fcca6f8fc admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.100473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba6e3a-7355-4a37-a431-2bc9481db755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.101072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6e3a-7355-4a37-a431-2bc9481db755 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.101353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6e3a-7355-4a37-a431-2bc9481db755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.101628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6e3a-7355-4a37-a431-2bc9481db755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.102315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba6e3a-7355-4a37-a431-2bc9481db755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.104432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba6e3a-7355-4a37-a431-2bc9481db755 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.114776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 786/1583] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.115911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdbb0-041b-47d4-94ff-e70dc2c59089 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.116190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdbb0-041b-47d4-94ff-e70dc2c59089 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.116486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cdbb0-041b-47d4-94ff-e70dc2c59089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.143121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-279cdbb0-041b-47d4-94ff-e70dc2c59089 None None] MFA Rules not processed for user `6f49507b26104b318153db0cdf993740`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:12.144314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-279cdbb0-041b-47d4-94ff-e70dc2c59089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:12.206415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 787/1584] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:12.207158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdbb0-041b-47d4-94ff-e70dc2c59089 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.217605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cdbb0-041b-47d4-94ff-e70dc2c59089 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.220004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 798/1585] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:12 2022] POST /identity/v3/users => generated 303 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:12.220004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.225610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.226405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.226855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.227028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.227801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.230256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.231142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff706a-96a5-430a-b67b-c8254abeb98c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.236710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.237449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.237667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.237946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.238674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.240690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.245952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 788/1586] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.245952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.256305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8f888-806c-475d-94f7-306c8a7e650e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.261315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.261909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.262186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.262456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.263139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.264746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.274545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 789/1587] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.275329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.284469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20a58e-0b87-4356-a35e-ccfa534f6f52 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.289004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.289591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.289880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.290164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.290850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.293738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.299069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:12.304328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 790/1588] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.305293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.315213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24bc66-eb83-4bf1-b5e1-87ed8bcb2fd4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.319750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.320313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.326690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.326690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.326690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.326690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.332603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 791/1589] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.333437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.344944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37afdb3-6c56-469c-86d7-8e73cd480508 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.349788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.350435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.350715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.350993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] PATH_INFO: `/v3/projects/470abc29b85043c0aed776bc9b529970/users/b0a98fbbafb348b1b5c937094ac3eb2d/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.354991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 799/1590] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:12 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:12.355829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.365996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f990303-4562-4b53-8360-9f5e7cabcc6c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.368719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=470abc29b85043c0aed776bc9b529970, user_id=b0a98fbbafb348b1b5c937094ac3eb2d, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.370630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:13:12.370630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:13:12.370630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:13:12.371612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72afbae-27a4-4b9a-b97c-27d37e668b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.372226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72afbae-27a4-4b9a-b97c-27d37e668b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.372549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72afbae-27a4-4b9a-b97c-27d37e668b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.372673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.372865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72afbae-27a4-4b9a-b97c-27d37e668b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.373535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72afbae-27a4-4b9a-b97c-27d37e668b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.375096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72afbae-27a4-4b9a-b97c-27d37e668b8f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.385701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 800/1591] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.386937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde2f92-31aa-4cb7-a89a-5abc689b0f8a None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.387219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde2f92-31aa-4cb7-a89a-5abc689b0f8a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.387490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde2f92-31aa-4cb7-a89a-5abc689b0f8a None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.388874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 801/1592] 10.222.0.47 () {54 vars in 962 bytes} [Wed Jul 27 20:13:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:12.389446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde2f92-31aa-4cb7-a89a-5abc689b0f8a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.402092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde2f92-31aa-4cb7-a89a-5abc689b0f8a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.406596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.407142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.407421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.407694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.408363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.409956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.420289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 802/1593] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.421112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.431258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1cc3e-625d-4947-bbc7-982897cb6ab4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.435878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.436502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.436844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.437124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.437796 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.439377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.449301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 803/1594] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.450176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.459719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b483a-e20c-47fb-a5cf-cc3ac37366ce admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.464758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.465385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.465668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.465949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] PATH_INFO: `/v3/projects/2ff46475bc95496f93ad4ac4459d7cc5/users/77b0df6c83a84dbcb0fa5802c93dc6ae/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.480835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ff46475bc95496f93ad4ac4459d7cc5, user_id=77b0df6c83a84dbcb0fa5802c93dc6ae, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.483483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.486470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 792/1595] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:12 2022] PUT /identity/v3/projects/470abc29b85043c0aed776bc9b529970/users/b0a98fbbafb348b1b5c937094ac3eb2d/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:12.487092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.545956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8c08b-df98-4323-9d80-fea247104d2d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.550577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.551182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.551466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.551740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.552458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.554213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.564168 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 793/1596] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.566255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 804/1597] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:12 2022] PUT /identity/v3/projects/2ff46475bc95496f93ad4ac4459d7cc5/users/77b0df6c83a84dbcb0fa5802c93dc6ae/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:12.570764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.570869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.579923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621d3b3-c88f-4e81-b433-75ed5d703b0e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.631595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b250f8e-4ac0-41ce-9944-36f74577a8d2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.636999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a9498-c134-4dc6-8f99-a0f08181cad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.637711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9498-c134-4dc6-8f99-a0f08181cad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.637992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9498-c134-4dc6-8f99-a0f08181cad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.638192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.638313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9498-c134-4dc6-8f99-a0f08181cad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.638815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.638976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a9498-c134-4dc6-8f99-a0f08181cad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.639098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.639383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.640109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.640710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a9498-c134-4dc6-8f99-a0f08181cad5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.641773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.650607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 805/1598] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.654561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555939da-75ef-4e9b-9b21-9e1a8daba93f None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.654698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 794/1599] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:12.654802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555939da-75ef-4e9b-9b21-9e1a8daba93f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.654802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555939da-75ef-4e9b-9b21-9e1a8daba93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.654942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.662524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0868a-d5a9-4bf7-ba15-fe8852135be0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.667444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a03b0-b248-4f38-9433-f5ab08519613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.668117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a03b0-b248-4f38-9433-f5ab08519613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.668418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a03b0-b248-4f38-9433-f5ab08519613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.668693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a03b0-b248-4f38-9433-f5ab08519613 admin admin] PATH_INFO: `/v3/projects/1c48ed1c9e2a4a1b840228b1313751f3/users/088d9846b00944a38315bb3ac9384929/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.683767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a03b0-b248-4f38-9433-f5ab08519613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c48ed1c9e2a4a1b840228b1313751f3, user_id=088d9846b00944a38315bb3ac9384929, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.686419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a03b0-b248-4f38-9433-f5ab08519613 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.690099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-555939da-75ef-4e9b-9b21-9e1a8daba93f None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:12.691321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-555939da-75ef-4e9b-9b21-9e1a8daba93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:12.742779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:12.756978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 806/1600] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Jul 27 20:13:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:12.757840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555939da-75ef-4e9b-9b21-9e1a8daba93f None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.768040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555939da-75ef-4e9b-9b21-9e1a8daba93f None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.772650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ffa5b-f858-47b9-97c0-c5c560bb599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.773283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ffa5b-f858-47b9-97c0-c5c560bb599f admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.773570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ffa5b-f858-47b9-97c0-c5c560bb599f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.773849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ffa5b-f858-47b9-97c0-c5c560bb599f admin admin] PATH_INFO: `/v3/projects/4ae4977eae84450dbb1fae0a76b7d711/users/a2026010ec9b4029883b51c4d5059511/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.787398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 795/1601] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:12 2022] PUT /identity/v3/projects/1c48ed1c9e2a4a1b840228b1313751f3/users/088d9846b00944a38315bb3ac9384929/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:12.788687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c648-b4dc-469a-9e26-66f9167c645b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.788965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ffa5b-f858-47b9-97c0-c5c560bb599f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae4977eae84450dbb1fae0a76b7d711, user_id=a2026010ec9b4029883b51c4d5059511, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.789104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c648-b4dc-469a-9e26-66f9167c645b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.789285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2c648-b4dc-469a-9e26-66f9167c645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.792152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ffa5b-f858-47b9-97c0-c5c560bb599f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.810073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8fa2c648-b4dc-469a-9e26-66f9167c645b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:12.811265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8fa2c648-b4dc-469a-9e26-66f9167c645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:12.839997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:12.871292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 796/1602] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:12.872161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2c648-b4dc-469a-9e26-66f9167c645b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:12.881124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2c648-b4dc-469a-9e26-66f9167c645b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:12.885906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:12.888259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.888259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.888259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] PATH_INFO: `/v3/projects/a56bf911ebbf4dd1ac5ed4bb07a9f7b3/users/6cc262a3c3314be8a6ee620ebfa6fe85/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.901289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56bf911ebbf4dd1ac5ed4bb07a9f7b3, user_id=6cc262a3c3314be8a6ee620ebfa6fe85, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:12.903950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:12.912821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 807/1603] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:12 2022] PUT /identity/v3/projects/4ae4977eae84450dbb1fae0a76b7d711/users/a2026010ec9b4029883b51c4d5059511/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:12.914201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f6e2-9dd6-4c6d-9733-1cda2331771d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:12.914486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f6e2-9dd6-4c6d-9733-1cda2331771d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:12.914757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f6e2-9dd6-4c6d-9733-1cda2331771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:12.934131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-81a6f6e2-9dd6-4c6d-9733-1cda2331771d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:12.935318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-81a6f6e2-9dd6-4c6d-9733-1cda2331771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:12.993947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 808/1604] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:12.995605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6f6e2-9dd6-4c6d-9733-1cda2331771d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.005209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6f6e2-9dd6-4c6d-9733-1cda2331771d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.012180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 797/1605] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:12 2022] PUT /identity/v3/projects/a56bf911ebbf4dd1ac5ed4bb07a9f7b3/users/6cc262a3c3314be8a6ee620ebfa6fe85/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:13.013121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.063948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923916a-04ed-404c-a69d-09010b0600e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.064713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923916a-04ed-404c-a69d-09010b0600e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.064994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923916a-04ed-404c-a69d-09010b0600e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.065279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923916a-04ed-404c-a69d-09010b0600e1 admin admin] PATH_INFO: `/v3/projects/bb1c9ff76b2a441ab3732316671faa04/users/3895268169824945885676fe9afeca2c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.075574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ba80-4c8f-4e2e-a25c-d8195c24c5b9 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.079830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923916a-04ed-404c-a69d-09010b0600e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1c9ff76b2a441ab3732316671faa04, user_id=3895268169824945885676fe9afeca2c, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.080211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.084315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.084315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.084315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.084315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.084735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923916a-04ed-404c-a69d-09010b0600e1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.084865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.094417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 798/1606] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:13.095213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.165942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd14e53-7b11-4c5b-8a74-f22a038e69da admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.166915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 809/1607] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:12 2022] PUT /identity/v3/projects/bb1c9ff76b2a441ab3732316671faa04/users/3895268169824945885676fe9afeca2c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:13.168327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8261df-f4d1-416c-b3dd-3394f44b3550 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.168639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8261df-f4d1-416c-b3dd-3394f44b3550 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.168923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8261df-f4d1-416c-b3dd-3394f44b3550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.201150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3e8261df-f4d1-416c-b3dd-3394f44b3550 None None] MFA Rules not processed for user `088d9846b00944a38315bb3ac9384929`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:13.202478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3e8261df-f4d1-416c-b3dd-3394f44b3550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.217583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:13.240559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72516a-b8a9-4103-bac3-e87f780002a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.241188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72516a-b8a9-4103-bac3-e87f780002a4 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.241470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72516a-b8a9-4103-bac3-e87f780002a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.241746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72516a-b8a9-4103-bac3-e87f780002a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.245164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:13.269050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 810/1608] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:13.270174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8261df-f4d1-416c-b3dd-3394f44b3550 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.302131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72516a-b8a9-4103-bac3-e87f780002a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.304968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72516a-b8a9-4103-bac3-e87f780002a4 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.314664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 799/1609] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:13.315938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f88788-e17b-4d2a-b771-30b84e6d499a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.316167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f88788-e17b-4d2a-b771-30b84e6d499a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.316476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f88788-e17b-4d2a-b771-30b84e6d499a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.346766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8261df-f4d1-416c-b3dd-3394f44b3550 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.346917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-71f88788-e17b-4d2a-b771-30b84e6d499a None None] MFA Rules not processed for user `a2026010ec9b4029883b51c4d5059511`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:13.346917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-71f88788-e17b-4d2a-b771-30b84e6d499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.347299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f713-43a8-4e27-a624-4e47d19a097a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.347980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f713-43a8-4e27-a624-4e47d19a097a service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.348677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f713-43a8-4e27-a624-4e47d19a097a service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.349023 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f713-43a8-4e27-a624-4e47d19a097a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.419600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f713-43a8-4e27-a624-4e47d19a097a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.421121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f713-43a8-4e27-a624-4e47d19a097a service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.425096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 800/1610] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:13.425770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f88788-e17b-4d2a-b771-30b84e6d499a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.436680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 811/1611] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:13.437965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378fab-3aec-4455-856c-ec785dc55295 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.438342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378fab-3aec-4455-856c-ec785dc55295 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.438609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a378fab-3aec-4455-856c-ec785dc55295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.440733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f88788-e17b-4d2a-b771-30b84e6d499a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.445023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb3542-2d7d-4f75-a1e6-2ca3dadf8168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.445947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3542-2d7d-4f75-a1e6-2ca3dadf8168 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.446237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3542-2d7d-4f75-a1e6-2ca3dadf8168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.448059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb3542-2d7d-4f75-a1e6-2ca3dadf8168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.458687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3a378fab-3aec-4455-856c-ec785dc55295 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:13.459573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb3542-2d7d-4f75-a1e6-2ca3dadf8168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.460558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3a378fab-3aec-4455-856c-ec785dc55295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.461795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb3542-2d7d-4f75-a1e6-2ca3dadf8168 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.472519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 812/1612] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:13.472519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a378fab-3aec-4455-856c-ec785dc55295 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.481133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a378fab-3aec-4455-856c-ec785dc55295 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.483458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 801/1613] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:13.484675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3c5e-0849-41e4-8cb9-4b51f3c7d07f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.490116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3c5e-0849-41e4-8cb9-4b51f3c7d07f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.490116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3c5e-0849-41e4-8cb9-4b51f3c7d07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.498269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2456404-1ac1-4fcf-ad4a-5d08a925bb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.500855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2456404-1ac1-4fcf-ad4a-5d08a925bb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.501382 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2456404-1ac1-4fcf-ad4a-5d08a925bb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.502700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2456404-1ac1-4fcf-ad4a-5d08a925bb60 admin admin] PATH_INFO: `/v3/projects/3448cb63d9b741a7956a5fc844820111/users/23d2938afd97422795662ec0581d4fa0/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.517559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-385e3c5e-0849-41e4-8cb9-4b51f3c7d07f None None] MFA Rules not processed for user `3895268169824945885676fe9afeca2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:13.519398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-385e3c5e-0849-41e4-8cb9-4b51f3c7d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.521860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2456404-1ac1-4fcf-ad4a-5d08a925bb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3448cb63d9b741a7956a5fc844820111, user_id=23d2938afd97422795662ec0581d4fa0, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.524758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2456404-1ac1-4fcf-ad4a-5d08a925bb60 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.575357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:13.595455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 802/1614] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:13.598309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41205e7-890d-4b1b-a0b1-b99c66940194 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.598309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41205e7-890d-4b1b-a0b1-b99c66940194 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.598309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41205e7-890d-4b1b-a0b1-b99c66940194 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.599059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 803/1615] 10.222.0.47 () {54 vars in 962 bytes} [Wed Jul 27 20:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:13.599849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41205e7-890d-4b1b-a0b1-b99c66940194 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.601670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f41205e7-890d-4b1b-a0b1-b99c66940194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.615213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41205e7-890d-4b1b-a0b1-b99c66940194 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.623124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461dfc7-9dc8-4d38-a847-40664e051ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.623982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461dfc7-9dc8-4d38-a847-40664e051ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.628639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461dfc7-9dc8-4d38-a847-40664e051ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.628776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 813/1616] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:13 2022] PUT /identity/v3/projects/3448cb63d9b741a7956a5fc844820111/users/23d2938afd97422795662ec0581d4fa0/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:13.628883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461dfc7-9dc8-4d38-a847-40664e051ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.628985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66e76-3582-4edd-b8d4-9f88d55b02aa None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.628985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66e76-3582-4edd-b8d4-9f88d55b02aa None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.628985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66e76-3582-4edd-b8d4-9f88d55b02aa None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.628985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 814/1617] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 27 20:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:13.628985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66e76-3582-4edd-b8d4-9f88d55b02aa None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.643372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461dfc7-9dc8-4d38-a847-40664e051ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.645444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461dfc7-9dc8-4d38-a847-40664e051ef3 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.657471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 804/1618] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:13.659163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c68c7-7431-45b4-94f4-5c0e41cf84fd None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.659946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c68c7-7431-45b4-94f4-5c0e41cf84fd None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.660257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c68c7-7431-45b4-94f4-5c0e41cf84fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.690806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c70c68c7-7431-45b4-94f4-5c0e41cf84fd None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:13.692039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c70c68c7-7431-45b4-94f4-5c0e41cf84fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.705210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66e76-3582-4edd-b8d4-9f88d55b02aa None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.710381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.711132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.711304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.711584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.712280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.714283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.719835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:13.765152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 805/1619] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:13.770411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dcc76-1838-44dc-bd45-6d5eb15fa49f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.770411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dcc76-1838-44dc-bd45-6d5eb15fa49f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.770411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561dcc76-1838-44dc-bd45-6d5eb15fa49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.789980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-561dcc76-1838-44dc-bd45-6d5eb15fa49f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:13.791370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-561dcc76-1838-44dc-bd45-6d5eb15fa49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.803499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 806/1620] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:13.803499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f83ea-fb6e-41da-8b96-d74d27732443 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.803499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f83ea-fb6e-41da-8b96-d74d27732443 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.803499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f83ea-fb6e-41da-8b96-d74d27732443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.827644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-945f83ea-fb6e-41da-8b96-d74d27732443 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:13.829307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-945f83ea-fb6e-41da-8b96-d74d27732443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:13.855841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 807/1621] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:13.856832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f83ea-fb6e-41da-8b96-d74d27732443 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.872243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f83ea-fb6e-41da-8b96-d74d27732443 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.878806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.879172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:13.879455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:13.879726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:13.905162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 815/1622] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:13 2022] POST /identity/v3/projects => generated 405 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:13.905162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.960014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:13.964606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:13.976496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 808/1623] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:13.977672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:13.989828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48385edf-cb6d-47c5-9c62-0e7ecd237c23 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:13.999241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:13.999879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.000164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.000463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.098605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcb02a-a0cb-47d8-87cc-e795fe4b1a55 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:14.106464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:14.159206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.161148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.188491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 816/1624] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:14.191496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:14.200825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a9d67-8fda-443b-a4d5-cccc0eca8534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:14.201452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9d67-8fda-443b-a4d5-cccc0eca8534 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.201731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9d67-8fda-443b-a4d5-cccc0eca8534 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.202005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9d67-8fda-443b-a4d5-cccc0eca8534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.205770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa9eef-508f-4712-b7a1-afba8766d6be service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:14.215069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:14.234557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.234910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.235226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.237412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.241567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.294982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a9d67-8fda-443b-a4d5-cccc0eca8534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.295284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a9d67-8fda-443b-a4d5-cccc0eca8534 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.306507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 809/1625] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:14.307796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563bd71-157c-4f29-b157-fac81cbbb397 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.308077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563bd71-157c-4f29-b157-fac81cbbb397 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.308360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563bd71-157c-4f29-b157-fac81cbbb397 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.309776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 810/1626] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 27 20:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:14.316705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71915e-db4d-4716-817b-7ed1ff949420 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.316995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71915e-db4d-4716-817b-7ed1ff949420 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.317253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71915e-db4d-4716-817b-7ed1ff949420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.339649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-ff71915e-db4d-4716-817b-7ed1ff949420 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:14.341000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ff71915e-db4d-4716-817b-7ed1ff949420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:14.351851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 811/1627] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:13:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:14.378402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 817/1628] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:14 2022] POST /identity/v3/users => generated 309 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:14.378583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71915e-db4d-4716-817b-7ed1ff949420 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:14.379425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:14.398212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf3880-9b3d-40e2-a954-5661240b37d1 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:14.403087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71915e-db4d-4716-817b-7ed1ff949420 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:14.410769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:14.411141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:14.421843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.421843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.421843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.422141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.422141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.422141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.422356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.422356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.442684 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.444753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.449426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 818/1629] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:13:14.453872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:14.463997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 812/1630] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:14.477785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ec139-7091-4718-a3ee-1e1c7746bbc7 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:14.491638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:14.492341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.492692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.493036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] PATH_INFO: `/v3/projects/90da6493d3c34a46ac27c6a82e1d8151/users/2ff6f0af83744be5b9cb5c7df75f9c1b/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.535176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90da6493d3c34a46ac27c6a82e1d8151, user_id=2ff6f0af83744be5b9cb5c7df75f9c1b, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.538006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.643792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 819/1631] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:14 2022] PUT /identity/v3/projects/90da6493d3c34a46ac27c6a82e1d8151/users/2ff6f0af83744be5b9cb5c7df75f9c1b/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:14.644776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:14.661526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfed78-13aa-41e3-ace8-1e695bb8d06a service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:14.762098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3a989-27cb-4ad1-ac67-128105c01a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:14.763268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3a989-27cb-4ad1-ac67-128105c01a4a admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.763552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3a989-27cb-4ad1-ac67-128105c01a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.763827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3a989-27cb-4ad1-ac67-128105c01a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.764736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3a989-27cb-4ad1-ac67-128105c01a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.767767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3a989-27cb-4ad1-ac67-128105c01a4a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.781495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 813/1632] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:14.782912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:14.795751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6192c-a850-44e0-a053-df8f5db2fb6a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:14.801617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5a3cd-0956-4ddf-a7b6-8607694406e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:14.802470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5a3cd-0956-4ddf-a7b6-8607694406e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.802945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5a3cd-0956-4ddf-a7b6-8607694406e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.803337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5a3cd-0956-4ddf-a7b6-8607694406e2 admin admin] PATH_INFO: `/v3/projects/90da6493d3c34a46ac27c6a82e1d8151/users/2ff6f0af83744be5b9cb5c7df75f9c1b/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.806298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5a3cd-0956-4ddf-a7b6-8607694406e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90da6493d3c34a46ac27c6a82e1d8151, user_id=2ff6f0af83744be5b9cb5c7df75f9c1b, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:14.809461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5a3cd-0956-4ddf-a7b6-8607694406e2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:14.821992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 820/1633] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:14 2022] PUT /identity/v3/projects/90da6493d3c34a46ac27c6a82e1d8151/users/2ff6f0af83744be5b9cb5c7df75f9c1b/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:14.831957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638907fd-91a5-45ea-897d-192d49623bfa None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:14.832238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638907fd-91a5-45ea-897d-192d49623bfa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:14.832579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638907fd-91a5-45ea-897d-192d49623bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:14.861392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-638907fd-91a5-45ea-897d-192d49623bfa None None] MFA Rules not processed for user `2ff6f0af83744be5b9cb5c7df75f9c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:14.862602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-638907fd-91a5-45ea-897d-192d49623bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:14.877118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:14.947590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 814/1634] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:15.365893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1b3ca-caa8-4842-a263-c8b69fd3e023 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:15.366468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1b3ca-caa8-4842-a263-c8b69fd3e023 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:15.366637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1b3ca-caa8-4842-a263-c8b69fd3e023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:15.417990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-62f1b3ca-caa8-4842-a263-c8b69fd3e023 None None] MFA Rules not processed for user `b0a98fbbafb348b1b5c937094ac3eb2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:15.419556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-62f1b3ca-caa8-4842-a263-c8b69fd3e023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:15.444149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:15.532489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638907fd-91a5-45ea-897d-192d49623bfa None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:15.539552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 821/1635] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:15.555300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1b3ca-caa8-4842-a263-c8b69fd3e023 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:15.644442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638907fd-91a5-45ea-897d-192d49623bfa None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:15.653375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:15.654029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:15.654309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:15.654581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:15.668074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1b3ca-caa8-4842-a263-c8b69fd3e023 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:15.675582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:15.676404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:15.677514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:15.678763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:15.679231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:15.680810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:15.681940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:15.683831 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:15.707456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 815/1636] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:15 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:15.717851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:15.835473 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 822/1637] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:15 2022] POST /identity/v3/projects => generated 405 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:15.836411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:15.937757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd99344-fc14-4311-bcec-fcd4a7bb114a service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:16.036195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e862d0-fb0b-4504-a6bf-99127ce17494 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:16.044673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:16.064111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:16.064961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:16.065243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:16.065570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:16.066407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:16.072956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:16.074632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:16.075278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:16.075562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:16.075839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:16.170967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:16.173362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:16.191085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 816/1638] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:16.211304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:16.247174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 823/1639] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:15 2022] POST /identity/v3/users => generated 309 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:16.250476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:16.274111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6da90-ca3b-4af3-9ead-0ad31405c63f service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:16.277815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3675c2-5e00-421c-b537-1c29c348a3a8 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:16.284349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:16.284929 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:16.285513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:16.285636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:16.285823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:16.286111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:16.286447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:16.286800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:16.287042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] PATH_INFO: `/v3/users/3895268169824945885676fe9afeca2c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:16.288491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:16.289995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3895268169824945885676fe9afeca2c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:16.309726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: WARNING py.warnings [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:13:16.309726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: warnings.warn(msg) Jul 27 20:13:16.309726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: Jul 27 20:13:16.596843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:16.599187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 824/1640] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:16.599187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:16.599187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b09dc-1c28-4979-9074-7fd370fcc42b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:16.599187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:16.606042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:16.606042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:16.606042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] PATH_INFO: `/v3/projects/34cc584a026b4659b34ceecead099673/users/0e67bd2d2b8c4dde91c157d96ec00d1a/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:16.606042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cc584a026b4659b34ceecead099673, user_id=0e67bd2d2b8c4dde91c157d96ec00d1a, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:16.606042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:16.609143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:16.609143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 825/1641] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:16 2022] PUT /identity/v3/projects/34cc584a026b4659b34ceecead099673/users/0e67bd2d2b8c4dde91c157d96ec00d1a/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:16.609143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:16.742697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56338ccc-2b64-4293-8ec8-acd469846588 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:16.757609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:16.758890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:16.759982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3895268169824945885676fe9afeca2c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:16.760279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3895268169824945885676fe9afeca2c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:16.760498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:16.761129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] PATH_INFO: `/v3/users/a2026010ec9b4029883b51c4d5059511` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:16.762678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2026010ec9b4029883b51c4d5059511)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:16.766215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:16.862812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3895268169824945885676fe9afeca2c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:16.911549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3895268169824945885676fe9afeca2c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:16.912975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3895268169824945885676fe9afeca2c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:16.950911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 817/1642] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:16 2022] DELETE /identity/v3/users/3895268169824945885676fe9afeca2c => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:16.952104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:16.953756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:17.035590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2026010ec9b4029883b51c4d5059511'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:17.036418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2026010ec9b4029883b51c4d5059511'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:17.115566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c501a3-4c5f-4dd1-95c9-ad73d8482144 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:17.122546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd422f3-3fc9-4f77-a614-009376e2f37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:17.123264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd422f3-3fc9-4f77-a614-009376e2f37b admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:17.123637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd422f3-3fc9-4f77-a614-009376e2f37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:17.123915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd422f3-3fc9-4f77-a614-009376e2f37b admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:17.124722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd422f3-3fc9-4f77-a614-009376e2f37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:17.126264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2026010ec9b4029883b51c4d5059511'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:17.126650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd422f3-3fc9-4f77-a614-009376e2f37b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:17.128509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:17.143350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 818/1643] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:17.144823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:17.145433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:17.145433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:17.163358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2026010ec9b4029883b51c4d5059511'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:17.176922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d3cb140-1a44-4d8f-bec8-9036c3871c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2026010ec9b4029883b51c4d5059511'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:17.179350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:17.180914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:17.193127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 819/1644] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:17.194740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:17.195560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:17.200328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 826/1645] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:16 2022] DELETE /identity/v3/users/a2026010ec9b4029883b51c4d5059511 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:17.205841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:17.206398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:17.208005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:17.214141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e023d-ef3a-428e-b1f0-9868c2b28586 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:17.262917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:17.266131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:17.333807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:17.335233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:17.337617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:17.338074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] PATH_INFO: `/v3/projects/34cc584a026b4659b34ceecead099673/users/0e67bd2d2b8c4dde91c157d96ec00d1a/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:17.340316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cc584a026b4659b34ceecead099673, user_id=0e67bd2d2b8c4dde91c157d96ec00d1a, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:17.350968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:17.368283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 820/1646] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:17 2022] PUT /identity/v3/projects/34cc584a026b4659b34ceecead099673/users/0e67bd2d2b8c4dde91c157d96ec00d1a/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:17.369946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:17.375866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:17.415277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 827/1647] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:17.472489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:17.473848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:17.589303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9545be-12ad-4bb2-81e5-03f60a9507db admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:17.606705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:17.607362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:17.610825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:17.612990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:17.653165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868cf0e-4607-454e-9cc2-7162431ca1bc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:17.662953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:17.665742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:17.666035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:17.668561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:17.742468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:17.745651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:17.763991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 821/1648] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:17.799902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:17.801703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:17.813572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 828/1649] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:18.352951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:19.981159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:19.982665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:20.006104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d8823-13db-4904-bde4-ae188e34aaa3 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:20.033103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:20.033783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:20.034296 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:20.034784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] PATH_INFO: `/v3/projects/bb1c9ff76b2a441ab3732316671faa04` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:20.036233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb1c9ff76b2a441ab3732316671faa04)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:20.038948 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:20.072934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:20.074391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:20.096636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30efb6c-bd1b-42ec-927e-3e11bd28652b service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:20.117742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:20.118781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:20.119973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:20.124000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] PATH_INFO: `/v3/users/088d9846b00944a38315bb3ac9384929` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:20.125982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=088d9846b00944a38315bb3ac9384929)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:20.128961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:20.193174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:20.230547 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb1c9ff76b2a441ab3732316671faa04', 'request_id': 'req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.246610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb1c9ff76b2a441ab3732316671faa04', 'request_id': 'req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.254209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb1c9ff76b2a441ab3732316671faa04', 'request_id': 'req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.355706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] The token cache is being invalidate because project bb1c9ff76b2a441ab3732316671faa04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:20.356008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.356736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.358314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 822/1650] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:19 2022] DELETE /identity/v3/projects/bb1c9ff76b2a441ab3732316671faa04 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:20.363503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:20.363503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:20.470125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088d9846b00944a38315bb3ac9384929'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.470860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088d9846b00944a38315bb3ac9384929'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.506366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6767a57-c2ee-4543-9ecf-98ae2d8e77df admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:20.514276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:20.519943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:20.519943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:20.519943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] PATH_INFO: `/v3/projects/4ae4977eae84450dbb1fae0a76b7d711` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:20.519943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae4977eae84450dbb1fae0a76b7d711)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:20.519943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:20.588823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088d9846b00944a38315bb3ac9384929'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.609617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088d9846b00944a38315bb3ac9384929'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.622403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088d9846b00944a38315bb3ac9384929'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.633108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 829/1651] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:20 2022] DELETE /identity/v3/users/088d9846b00944a38315bb3ac9384929 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:20.634921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:20.636697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:20.731051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:20.746402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae4977eae84450dbb1fae0a76b7d711', 'request_id': 'req-d10be53a-09ee-400f-bd77-a488b52871e0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.759600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae4977eae84450dbb1fae0a76b7d711', 'request_id': 'req-d10be53a-09ee-400f-bd77-a488b52871e0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.770708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d48eb6-9483-47d9-b14e-8c860bbc9315 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:20.772165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae4977eae84450dbb1fae0a76b7d711', 'request_id': 'req-d10be53a-09ee-400f-bd77-a488b52871e0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.870882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] The token cache is being invalidate because project 4ae4977eae84450dbb1fae0a76b7d711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:20.872010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.873148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d10be53a-09ee-400f-bd77-a488b52871e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:20.880255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 823/1652] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:20 2022] DELETE /identity/v3/projects/4ae4977eae84450dbb1fae0a76b7d711 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:20.882372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8f09-92e9-450c-be24-cfab1ab51204 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:20.883230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8f09-92e9-450c-be24-cfab1ab51204 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:20.883908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e8f09-92e9-450c-be24-cfab1ab51204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:20.909922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:20.910650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:20.910866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:20.911146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] PATH_INFO: `/v3/users/2ff6f0af83744be5b9cb5c7df75f9c1b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:20.917057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ff6f0af83744be5b9cb5c7df75f9c1b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:20.919255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:20.924710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a54e8f09-92e9-450c-be24-cfab1ab51204 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:20.926481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a54e8f09-92e9-450c-be24-cfab1ab51204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:20.953807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 824/1653] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:20.957153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:20.957762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:20.958230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:20.993463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:20.995949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:21.006227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 825/1654] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:21.007555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:21.010079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:21.150055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8289e-168f-41a1-859f-9b68907652fd None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:21.154983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ff6f0af83744be5b9cb5c7df75f9c1b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:21.156089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ff6f0af83744be5b9cb5c7df75f9c1b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:21.161192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ff129-ee4f-4401-b791-085d10291adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:21.162454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff129-ee4f-4401-b791-085d10291adb service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:21.164004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff129-ee4f-4401-b791-085d10291adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:21.164538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ff129-ee4f-4401-b791-085d10291adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:21.179720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ff129-ee4f-4401-b791-085d10291adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:21.182040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ff129-ee4f-4401-b791-085d10291adb service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:21.197359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 826/1655] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:21.199085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148da48-7873-43ef-a679-77d0debc719f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:21.199590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148da48-7873-43ef-a679-77d0debc719f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:21.200011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148da48-7873-43ef-a679-77d0debc719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:21.226279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c148da48-7873-43ef-a679-77d0debc719f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:21.227925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c148da48-7873-43ef-a679-77d0debc719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:21.234239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ff6f0af83744be5b9cb5c7df75f9c1b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:21.256807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ff6f0af83744be5b9cb5c7df75f9c1b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:21.256961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 827/1656] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:21.259037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148da48-7873-43ef-a679-77d0debc719f None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:21.268965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ff6f0af83744be5b9cb5c7df75f9c1b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:21.279862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148da48-7873-43ef-a679-77d0debc719f None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:21.284431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 830/1657] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:20 2022] DELETE /identity/v3/users/2ff6f0af83744be5b9cb5c7df75f9c1b => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:21.477244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759993-31ee-4124-8c03-0ea8ebc8b1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:21.479712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759993-31ee-4124-8c03-0ea8ebc8b1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:21.480222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759993-31ee-4124-8c03-0ea8ebc8b1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:21.480667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759993-31ee-4124-8c03-0ea8ebc8b1f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:21.481576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759993-31ee-4124-8c03-0ea8ebc8b1f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:21.482922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759993-31ee-4124-8c03-0ea8ebc8b1f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:21.484987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759993-31ee-4124-8c03-0ea8ebc8b1f5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:21.501842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 828/1658] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:21.506389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:21.510187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:21.563784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec82d6-1a87-451e-b844-946c0edd5e68 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:21.580330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:21.581307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:21.581671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:21.583683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:21.588113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:21.599832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:21.620925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f6203-e314-4d13-b080-b6ab2088fd87 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:21.621219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f6203-e314-4d13-b080-b6ab2088fd87 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:21.621503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f6203-e314-4d13-b080-b6ab2088fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:21.685463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8c5f6203-e314-4d13-b080-b6ab2088fd87 None None] MFA Rules not processed for user `0e67bd2d2b8c4dde91c157d96ec00d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:21.686719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8c5f6203-e314-4d13-b080-b6ab2088fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:21.746593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 831/1659] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:21 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:21.756250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:21.898590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 829/1660] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:21.917123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a9518-d7af-4738-ab07-f6af4d157699 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:21.923652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:21.927920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:21.927920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:21.927920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:21.927920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:21.930787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:22.014941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 832/1661] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:21 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:22.023553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f6203-e314-4d13-b080-b6ab2088fd87 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:22.039361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f6203-e314-4d13-b080-b6ab2088fd87 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:22.055780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:22.057306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:22.057767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:22.058246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:22.059130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:22.060969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:22.088469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 830/1662] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:22.090199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:22.107205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f22521-db72-46bd-a25b-5e2368b5adb3 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:22.125135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7283aa2-483b-4f9d-a966-b8c7edcd3d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:22.126120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283aa2-483b-4f9d-a966-b8c7edcd3d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:22.126582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283aa2-483b-4f9d-a966-b8c7edcd3d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:22.127030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7283aa2-483b-4f9d-a966-b8c7edcd3d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:22.127932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283aa2-483b-4f9d-a966-b8c7edcd3d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:22.129945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7283aa2-483b-4f9d-a966-b8c7edcd3d98 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:22.162326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 833/1663] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:22.167349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:22.182851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff35159-a8b7-4e95-831c-2df960af45cd admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:22.200003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:22.201112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:22.201599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:22.202100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] PATH_INFO: `/v3/projects/4e930d1e53c54b659a825aa1784bae67/users/2b2516350efd483faa420d25463ce320/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:22.222534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e930d1e53c54b659a825aa1784bae67, user_id=2b2516350efd483faa420d25463ce320, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:22.225521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:22.230003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:22.345234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 831/1664] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:22 2022] PUT /identity/v3/projects/4e930d1e53c54b659a825aa1784bae67/users/2b2516350efd483faa420d25463ce320/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:22.357038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:22.357441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:22.357780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:22.405635 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] MFA Rules not processed for user `2b2516350efd483faa420d25463ce320`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:22.408810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:22.522033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 834/1665] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:22.526794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:22.554277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:22.555881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:22.612658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:22.674224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a5f99-fb36-48eb-905e-0bfd7fc04abc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:22.688488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:22.690050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:22.690457 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:22.691722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] PATH_INFO: `/v3/projects/90da6493d3c34a46ac27c6a82e1d8151` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:22.697330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90da6493d3c34a46ac27c6a82e1d8151)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:22.698756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: WARNING py.warnings [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:13:22.698756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: warnings.warn(msg) Jul 27 20:13:22.698756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: Jul 27 20:13:22.703977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:22.754570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455faa7c-df10-427c-87f5-3584952b1dcb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:22.764673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:22.768316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:22.768535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:22.768859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:22.769558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:22.773035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:22.932637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 832/1666] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:22 2022] POST /identity/v3/projects => generated 381 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:22.933957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:22.944513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:22.953462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90da6493d3c34a46ac27c6a82e1d8151', 'request_id': 'req-58993466-e405-48df-954d-88a8aecf9f77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:22.997899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90da6493d3c34a46ac27c6a82e1d8151', 'request_id': 'req-58993466-e405-48df-954d-88a8aecf9f77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.011650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90da6493d3c34a46ac27c6a82e1d8151', 'request_id': 'req-58993466-e405-48df-954d-88a8aecf9f77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.092734 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe427bc-8184-41c9-b087-f79972a290e6 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:23.100020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:23.101011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.101360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.101693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] PATH_INFO: `/v3/projects/1c48ed1c9e2a4a1b840228b1313751f3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.103008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c48ed1c9e2a4a1b840228b1313751f3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:23.107018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:23.164909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] The token cache is being invalidate because project 90da6493d3c34a46ac27c6a82e1d8151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:23.165388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.166434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.173470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 835/1667] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:22 2022] DELETE /identity/v3/projects/90da6493d3c34a46ac27c6a82e1d8151 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:23.174716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:23.176469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:23.246972 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c48ed1c9e2a4a1b840228b1313751f3', 'request_id': 'req-a0d90e06-6fe4-4a32-8476-fe828730aaa1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.256943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c48ed1c9e2a4a1b840228b1313751f3', 'request_id': 'req-a0d90e06-6fe4-4a32-8476-fe828730aaa1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.274504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c48ed1c9e2a4a1b840228b1313751f3', 'request_id': 'req-a0d90e06-6fe4-4a32-8476-fe828730aaa1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.308101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58993466-e405-48df-954d-88a8aecf9f77 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:23.366374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] The token cache is being invalidate because project 1c48ed1c9e2a4a1b840228b1313751f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:23.366671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.367348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a0d90e06-6fe4-4a32-8476-fe828730aaa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:23.373485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 833/1668] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:22 2022] DELETE /identity/v3/projects/1c48ed1c9e2a4a1b840228b1313751f3 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:23.373485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab7b2f-0031-444d-a2f2-4af8effb57d6 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.373485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab7b2f-0031-444d-a2f2-4af8effb57d6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.373485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab7b2f-0031-444d-a2f2-4af8effb57d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.415446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff8fc8-eb55-4b89-b81a-2e01a6add2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:23.416153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff8fc8-eb55-4b89-b81a-2e01a6add2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.416458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff8fc8-eb55-4b89-b81a-2e01a6add2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.418185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-baab7b2f-0031-444d-a2f2-4af8effb57d6 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:23.418668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff8fc8-eb55-4b89-b81a-2e01a6add2b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.419530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-baab7b2f-0031-444d-a2f2-4af8effb57d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:23.420454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff8fc8-eb55-4b89-b81a-2e01a6add2b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:23.435008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff8fc8-eb55-4b89-b81a-2e01a6add2b4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:23.457046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 834/1669] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:23.461291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d5dc-bea2-47fd-801d-2b65d97ae0bc None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.467219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d5dc-bea2-47fd-801d-2b65d97ae0bc None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.467720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6d5dc-bea2-47fd-801d-2b65d97ae0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.500927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8db6d5dc-bea2-47fd-801d-2b65d97ae0bc None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:23.502431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8db6d5dc-bea2-47fd-801d-2b65d97ae0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:23.512151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 835/1670] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:23.513742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.515871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.516291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.545549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:23.547172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:23.564605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 836/1671] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:23 2022] POST /identity/v3/users => generated 297 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:23.564605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d632258-f56e-4f10-a3b7-d6d279748975 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.564605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d632258-f56e-4f10-a3b7-d6d279748975 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.564605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d632258-f56e-4f10-a3b7-d6d279748975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.576041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 836/1672] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:23.577371 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:23.579162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:23.608192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ae7b8-2261-47d9-a8d7-80e26ba48fe3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:23.616603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3d632258-f56e-4f10-a3b7-d6d279748975 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:23.622297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3d632258-f56e-4f10-a3b7-d6d279748975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:23.623100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:23.623930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.624538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.624952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.625818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:23.627891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:23.635264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 837/1673] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:23.636131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d632258-f56e-4f10-a3b7-d6d279748975 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:23.647994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d632258-f56e-4f10-a3b7-d6d279748975 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:23.655058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:23.655559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 837/1674] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:23.655756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.655970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.656254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.657513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:23.657513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:23.659069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:23.659411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:23.675414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 838/1675] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:23.676733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:23.687935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a2d00-4520-4f14-bc4b-4e46e20ce8d7 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:23.688118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03d345-dec5-43d0-9788-48a737898397 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:23.694334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:23.695428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.695649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.695936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] PATH_INFO: `/v3/projects/3726e69ea04046eebe548b4167385ee6/users/90e0893b50904214bbc046fe3fe9722b/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.697277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:23.697907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.698216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.698465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.699178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:23.699486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:23.702488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:23.716826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3726e69ea04046eebe548b4167385ee6, user_id=90e0893b50904214bbc046fe3fe9722b, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:23.719518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:23.738305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 838/1676] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:23.739152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:23.758024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03c1ad-1d7f-4426-a28d-37084fd94335 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:23.773436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:23.774084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:23.774365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:23.774642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:23.775947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:23.779268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:23.854808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 839/1677] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:23 2022] PUT /identity/v3/projects/3726e69ea04046eebe548b4167385ee6/users/90e0893b50904214bbc046fe3fe9722b/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:23.855872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:23.963561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 839/1678] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:23 2022] POST /identity/v3/projects => generated 373 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:23.965249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.025869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728814e0-a243-44b2-a6b0-7528e6631657 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.108841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04115efe-9ed9-4bbc-8082-09044d4e6bb0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.115429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.116483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.116738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.117149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.118450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.121118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.133957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.135203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.135683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.136153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.137131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.139906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.141678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 840/1679] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:24.142622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.158583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d6803-4d3d-459b-bd6e-570d1857c685 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.172105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:24.175254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.176755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.177398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.181837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.257469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 840/1680] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:23 2022] POST /identity/v3/projects => generated 381 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:24.257469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.284501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 841/1681] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:24 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:24.289926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.306263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6f03d-f107-46df-93fd-40d1e978fbdf admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.406573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcd4ef-731a-4518-b601-35e0261b11ce admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.419455 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.419792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.420047 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.420327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] PATH_INFO: `/v3/projects/3726e69ea04046eebe548b4167385ee6/users/90e0893b50904214bbc046fe3fe9722b/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.425980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3726e69ea04046eebe548b4167385ee6, user_id=90e0893b50904214bbc046fe3fe9722b, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.433492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.442541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87b8d1-96c6-49fc-9974-7dd294084d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.443588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b8d1-96c6-49fc-9974-7dd294084d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.444155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b8d1-96c6-49fc-9974-7dd294084d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.444890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87b8d1-96c6-49fc-9974-7dd294084d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.445802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87b8d1-96c6-49fc-9974-7dd294084d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.448707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87b8d1-96c6-49fc-9974-7dd294084d2c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.466139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 841/1682] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:24 2022] PUT /identity/v3/projects/3726e69ea04046eebe548b4167385ee6/users/90e0893b50904214bbc046fe3fe9722b/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:24.466139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.575665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce875450-d5f2-4182-8b14-3851852b1b7a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.582397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c387d8-f886-44db-99a3-c7f3bfa6c93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.584002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c387d8-f886-44db-99a3-c7f3bfa6c93b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.584589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c387d8-f886-44db-99a3-c7f3bfa6c93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.585241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c387d8-f886-44db-99a3-c7f3bfa6c93b admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.597782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 842/1683] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:24 2022] POST /identity/v3/users => generated 291 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:24.599568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1b145-2939-4cc0-9524-618b7f641f0f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.600059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1b145-2939-4cc0-9524-618b7f641f0f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.600644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1b145-2939-4cc0-9524-618b7f641f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.601403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c387d8-f886-44db-99a3-c7f3bfa6c93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.606297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:24.607905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c387d8-f886-44db-99a3-c7f3bfa6c93b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.637093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 842/1684] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:24.640611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d355177-b34f-4091-a2d9-03ca465b9aa0 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.642627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d355177-b34f-4091-a2d9-03ca465b9aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.643141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d355177-b34f-4091-a2d9-03ca465b9aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.681869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-31a1b145-2939-4cc0-9524-618b7f641f0f None None] MFA Rules not processed for user `90e0893b50904214bbc046fe3fe9722b`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:24.684459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-31a1b145-2939-4cc0-9524-618b7f641f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:24.706808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3d355177-b34f-4091-a2d9-03ca465b9aa0 None None] MFA Rules not processed for user `77b0df6c83a84dbcb0fa5802c93dc6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:24.708662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3d355177-b34f-4091-a2d9-03ca465b9aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:24.817990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 843/1685] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:24.825736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1b145-2939-4cc0-9524-618b7f641f0f None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.860325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1b145-2939-4cc0-9524-618b7f641f0f None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.867375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.868018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.868302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.871278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.871592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.876997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.911217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 843/1687] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:24.911422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 844/1687] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:24.911422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.928857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc51200-9947-48a4-82e5-f73644775b56 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.929432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc51200-9947-48a4-82e5-f73644775b56 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.929724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc51200-9947-48a4-82e5-f73644775b56 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.932982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 844/1688] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:13:24 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:24.934565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc51200-9947-48a4-82e5-f73644775b56 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.937689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9bc51200-9947-48a4-82e5-f73644775b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:24.938071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8fe27-88e6-4b5e-bb3c-e8818c467264 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.947133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:24.948488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:24.948726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:24.949216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:24.949945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:24.951710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:24.970137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 845/1689] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:24.971629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:24.985282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7237034-b163-41aa-a894-d47a1577b4d5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:24.999500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36ecad-dadf-4fb9-9ace-d34fb9d94b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:25.000550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36ecad-dadf-4fb9-9ace-d34fb9d94b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.001232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36ecad-dadf-4fb9-9ace-d34fb9d94b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.002424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36ecad-dadf-4fb9-9ace-d34fb9d94b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.003387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36ecad-dadf-4fb9-9ace-d34fb9d94b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:25.006401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36ecad-dadf-4fb9-9ace-d34fb9d94b63 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:25.024714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 846/1690] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:25.027706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fff7fb-71f3-48c7-af9f-1eeac65bec44 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.028236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fff7fb-71f3-48c7-af9f-1eeac65bec44 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.028672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fff7fb-71f3-48c7-af9f-1eeac65bec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.073074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-89fff7fb-71f3-48c7-af9f-1eeac65bec44 None None] MFA Rules not processed for user `77b0df6c83a84dbcb0fa5802c93dc6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:25.080034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-89fff7fb-71f3-48c7-af9f-1eeac65bec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:25.089969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 847/1691] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:25.094400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa6beb-630c-4984-af00-10013f92a1d6 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.094690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa6beb-630c-4984-af00-10013f92a1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.095387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa6beb-630c-4984-af00-10013f92a1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.163008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-96fa6beb-630c-4984-af00-10013f92a1d6 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:25.167009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-96fa6beb-630c-4984-af00-10013f92a1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:25.175122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc51200-9947-48a4-82e5-f73644775b56 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:25.190081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:25.194427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:25.199862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.200572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.200680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.226607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:25.230285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:25.256923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 845/1692] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:25.263853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:25.297234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27626c89-fba0-485b-a177-fbc9f31d7ced service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:25.310214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:25.310906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.311177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.311451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] PATH_INFO: `/v3/projects/cded4b3b0fdd40688ff8cbda6121aa32/users/3f445dd5360444639684f0f7a213e2d6/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.354335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cded4b3b0fdd40688ff8cbda6121aa32, user_id=3f445dd5360444639684f0f7a213e2d6, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:25.357872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:25.364502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 848/1693] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:13:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:25.365341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa6beb-630c-4984-af00-10013f92a1d6 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:25.389896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa6beb-630c-4984-af00-10013f92a1d6 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:25.403975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567acddb-f49a-4c21-80dd-7d0ff181e975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:25.407815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acddb-f49a-4c21-80dd-7d0ff181e975 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.407815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acddb-f49a-4c21-80dd-7d0ff181e975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.407815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567acddb-f49a-4c21-80dd-7d0ff181e975 admin admin] PATH_INFO: `/v3/projects/4fcdb4d877064f7dbfd5c2dfb813d3d4/users/a0caf208424b463bb1aefb69311dd482/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.422350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567acddb-f49a-4c21-80dd-7d0ff181e975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fcdb4d877064f7dbfd5c2dfb813d3d4, user_id=a0caf208424b463bb1aefb69311dd482, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:25.425254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567acddb-f49a-4c21-80dd-7d0ff181e975 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:25.543998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 846/1695] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:25 2022] PUT /identity/v3/projects/cded4b3b0fdd40688ff8cbda6121aa32/users/3f445dd5360444639684f0f7a213e2d6/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:25.553316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:25.553415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 849/1695] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:25 2022] PUT /identity/v3/projects/4fcdb4d877064f7dbfd5c2dfb813d3d4/users/a0caf208424b463bb1aefb69311dd482/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:25.555497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8951-961b-4166-b5c4-cf9de477e32b None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.555610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:25.555683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8951-961b-4166-b5c4-cf9de477e32b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.555683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8951-961b-4166-b5c4-cf9de477e32b None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.556644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 850/1696] 10.222.0.47 () {54 vars in 961 bytes} [Wed Jul 27 20:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:25.557721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5fd2a-b05b-4c85-a0ec-e1ca9d9102ed None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.558069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5fd2a-b05b-4c85-a0ec-e1ca9d9102ed None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.558342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5fd2a-b05b-4c85-a0ec-e1ca9d9102ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.626535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-85a5fd2a-b05b-4c85-a0ec-e1ca9d9102ed None None] MFA Rules not processed for user `a0caf208424b463bb1aefb69311dd482`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:25.628234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-85a5fd2a-b05b-4c85-a0ec-e1ca9d9102ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:25.722429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222e58-04e4-49f8-9ed2-f7e045f31bf2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:25.733131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dc3e3-44df-4f6f-a98c-db3b541f9859 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:25.733590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc3e3-44df-4f6f-a98c-db3b541f9859 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.733870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc3e3-44df-4f6f-a98c-db3b541f9859 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.734140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc3e3-44df-4f6f-a98c-db3b541f9859 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.777085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 851/1697] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:25.781776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e6e5-0145-48b2-b687-162cb5859743 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.782070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e6e5-0145-48b2-b687-162cb5859743 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.782348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e6e5-0145-48b2-b687-162cb5859743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.851023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a587e6e5-0145-48b2-b687-162cb5859743 None None] MFA Rules not processed for user `3f445dd5360444639684f0f7a213e2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:25.852188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a587e6e5-0145-48b2-b687-162cb5859743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:25.896752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dc3e3-44df-4f6f-a98c-db3b541f9859 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:25.899575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dc3e3-44df-4f6f-a98c-db3b541f9859 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:25.932824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 847/1698] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:25.935956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373454e2-6d68-470d-9c2a-4281eab47d6c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:25.936819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373454e2-6d68-470d-9c2a-4281eab47d6c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:25.937106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373454e2-6d68-470d-9c2a-4281eab47d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:25.989621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-373454e2-6d68-470d-9c2a-4281eab47d6c None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:25.991444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 852/1699] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:25.992420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e6e5-0145-48b2-b687-162cb5859743 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:25.994843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-373454e2-6d68-470d-9c2a-4281eab47d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:26.102726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e6e5-0145-48b2-b687-162cb5859743 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:26.108129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:26.109146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:26.111645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:26.112333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:26.114770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:26.184126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:26.201227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 848/1700] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Jul 27 20:13:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:26.210437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373454e2-6d68-470d-9c2a-4281eab47d6c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:26.234487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:26.236160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:26.252022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 853/1701] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:26.252946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:26.465486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373454e2-6d68-470d-9c2a-4281eab47d6c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:26.487016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:26.487823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:26.490663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:26.492318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:26.493210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:26.498516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:26.501796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66289b-49b5-43bc-8c1c-5b4145200a97 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:26.508039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:26.508800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:26.509597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:26.509885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:26.510743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:26.513136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:26.629756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 849/1702] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:26 2022] POST /identity/v3/projects => generated 381 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:26.630803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:26.685502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 854/1703] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:26 2022] POST /identity/v3/projects => generated 373 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:26.687735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:26.756611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bfb14-e69e-482b-9d51-83ce87bbcd7f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:26.840784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca14499-9594-4d96-9045-1b7d527fc253 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:26.847259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:26.848160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:26.848629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:26.849046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:26.849874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:26.852288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:26.914904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:26.916332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:26.916759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:26.917342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:26.928082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:26.929931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:26.932142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:26.976361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 850/1704] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:26.977523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:27.015867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 855/1705] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:26 2022] POST /identity/v3/users => generated 297 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:27.020621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:27.030316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fcff8-764b-4e78-88a5-e426dd6447bb service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:27.031498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:27.082565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:27.083207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.083487 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.083766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.084476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:27.102924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:27.103304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96c3d-95ef-425a-b0e1-4d12b348a2d3 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:27.117097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:27.117773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.118066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.118353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] PATH_INFO: `/v3/users/b0a98fbbafb348b1b5c937094ac3eb2d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.120650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a98fbbafb348b1b5c937094ac3eb2d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:27.122619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:27.224668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 851/1706] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:26 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:27.225649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:27.259140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf99bcc-c7a3-4dfb-9cd8-20e7d40cf61a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:27.275324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50392f4-4c8a-408e-a089-6288ba776024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:27.275978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50392f4-4c8a-408e-a089-6288ba776024 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.276261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50392f4-4c8a-408e-a089-6288ba776024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.276574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50392f4-4c8a-408e-a089-6288ba776024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.279553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50392f4-4c8a-408e-a089-6288ba776024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:27.279553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50392f4-4c8a-408e-a089-6288ba776024 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:27.311754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 852/1707] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:27.314590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b6034-5380-4528-9296-bf9db6894a75 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.317011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b6034-5380-4528-9296-bf9db6894a75 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.317325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b6034-5380-4528-9296-bf9db6894a75 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.324346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 853/1708] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 27 20:13:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:27.325517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b6034-5380-4528-9296-bf9db6894a75 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:27.333273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bb1b6034-5380-4528-9296-bf9db6894a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:27.399785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a98fbbafb348b1b5c937094ac3eb2d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.401897 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a98fbbafb348b1b5c937094ac3eb2d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.404403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b6034-5380-4528-9296-bf9db6894a75 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:27.429499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:27.430232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.430509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.432431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] PATH_INFO: `/v3/users/6f49507b26104b318153db0cdf993740` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.434286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f49507b26104b318153db0cdf993740)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:27.436238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:27.483882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a98fbbafb348b1b5c937094ac3eb2d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.505393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a98fbbafb348b1b5c937094ac3eb2d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.506222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a98fbbafb348b1b5c937094ac3eb2d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.537655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 856/1709] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:27 2022] DELETE /identity/v3/users/b0a98fbbafb348b1b5c937094ac3eb2d => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:27.541967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:27.543569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:27.666786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f49507b26104b318153db0cdf993740'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.668498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f49507b26104b318153db0cdf993740'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.669554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e6943-f309-4d7f-8796-583c4a032241 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:27.676613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:27.678708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.678997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.679275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.680043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:27.682388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:27.696757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 857/1710] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:27.698167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:27.700991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:27.717711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9121f5-ac68-425b-905f-e9357a9bdfdf admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:27.723319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfacfc8-afec-43fa-b649-adcbdc31b46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:27.724328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfacfc8-afec-43fa-b649-adcbdc31b46c admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.724838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfacfc8-afec-43fa-b649-adcbdc31b46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.725290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfacfc8-afec-43fa-b649-adcbdc31b46c admin admin] PATH_INFO: `/v3/projects/f120fb97f91f488bb350faa71094e5e9/users/019c3215257d48c1bd3848db809774db/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.745331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfacfc8-afec-43fa-b649-adcbdc31b46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f120fb97f91f488bb350faa71094e5e9, user_id=019c3215257d48c1bd3848db809774db, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:27.748047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfacfc8-afec-43fa-b649-adcbdc31b46c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:27.772083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f49507b26104b318153db0cdf993740'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.792063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f49507b26104b318153db0cdf993740'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.807101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-68ef4aa9-4ce9-466c-834f-90e6cbd03c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f49507b26104b318153db0cdf993740'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:27.816736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 854/1711] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:27 2022] DELETE /identity/v3/users/6f49507b26104b318153db0cdf993740 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:27.818476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.818970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.819258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.836595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 858/1712] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:27 2022] PUT /identity/v3/projects/f120fb97f91f488bb350faa71094e5e9/users/019c3215257d48c1bd3848db809774db/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:27.838122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1bd0b-fca8-488f-b84a-5faaece5a4d2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:27.838403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1bd0b-fca8-488f-b84a-5faaece5a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:27.838684 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1bd0b-fca8-488f-b84a-5faaece5a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:27.874108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] MFA Rules not processed for user `54aedfbb64c54c589003b335fde2a17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:27.879077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:27.921174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d0f1bd0b-fca8-488f-b84a-5faaece5a4d2 None None] MFA Rules not processed for user `6cc262a3c3314be8a6ee620ebfa6fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:27.921174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d0f1bd0b-fca8-488f-b84a-5faaece5a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.009996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 855/1713] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:13:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:28.010963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:28.015335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.087701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:28.111984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 859/1714] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:28.113978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f61227-b843-4701-a994-36f0661f51ec None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:28.114441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f61227-b843-4701-a994-36f0661f51ec None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:28.114934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f61227-b843-4701-a994-36f0661f51ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:28.137426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517b771-143b-49d0-a736-e3e11139e732 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:28.139516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-83f61227-b843-4701-a994-36f0661f51ec None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:28.140113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-83f61227-b843-4701-a994-36f0661f51ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.143163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:28.143989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:28.144346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:28.144861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] PATH_INFO: `/v3/projects/16b8dd84d08e4bacb20d2b8b5387407b/users/d078d883d5f34714bb233797553a484e/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:28.169164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b8dd84d08e4bacb20d2b8b5387407b, user_id=d078d883d5f34714bb233797553a484e, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:28.172721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:28.175295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 860/1715] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:28.176435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f61227-b843-4701-a994-36f0661f51ec None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:28.186047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-83f61227-b843-4701-a994-36f0661f51ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.209368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f61227-b843-4701-a994-36f0661f51ec None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:28.216980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:28.217979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:28.218503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:28.219127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:28.219994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:28.222929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:28.259083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 861/1716] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:28.260452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:28.262657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.280650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09eda6b-f354-46f7-a382-1632075395ff admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:28.291074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 856/1717] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:28 2022] PUT /identity/v3/projects/16b8dd84d08e4bacb20d2b8b5387407b/users/d078d883d5f34714bb233797553a484e/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:28.292898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:28.297664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.315890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:28.362565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:28.363557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:28.364047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:28.364575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] PATH_INFO: `/v3/projects/470abc29b85043c0aed776bc9b529970` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:28.373964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=470abc29b85043c0aed776bc9b529970)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:28.373964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:28.395057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325ec2f-a562-4991-9aec-169483ce7536 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:28.402632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:28.403748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:28.403748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:28.403936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:28.416526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:28.418138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:28.428974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 857/1718] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:28.429682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:28.432291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.546293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470abc29b85043c0aed776bc9b529970', 'request_id': 'req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:28.572755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470abc29b85043c0aed776bc9b529970', 'request_id': 'req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:28.600733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470abc29b85043c0aed776bc9b529970', 'request_id': 'req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:28.645696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65808cd6-ea55-4808-89a8-9b0a8380417b service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:28.699713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] The token cache is being invalidate because project 470abc29b85043c0aed776bc9b529970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:28.704417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:28.704417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:28.704989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 862/1719] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:28 2022] DELETE /identity/v3/projects/470abc29b85043c0aed776bc9b529970 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:28.709193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:28.713610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.832267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:28.833185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:28.833594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:28.834012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:28.839442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.883695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198762c-3ed0-4baf-b219-cb8b7ebdbc4a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:28.891344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:28.892007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:28.892293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:28.892627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:28.894708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:28.952481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:28.953169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:28.965532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 858/1720] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:28.966405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:28.983838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.023720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:29.027704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:29.056470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 863/1721] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:29.059056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:29.063813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.092606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf764e1-9c85-4e9f-aefa-23e0198192ca service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:29.111223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:29.111880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.112156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.112452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.113158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:29.114974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:29.117054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0dcb-c9a4-4c3d-97e2-0700e7ac1a5c service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:29.126079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:29.127519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.127812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.128093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] PATH_INFO: `/v3/projects/f120fb97f91f488bb350faa71094e5e9/users/019c3215257d48c1bd3848db809774db/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.130607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f120fb97f91f488bb350faa71094e5e9, user_id=019c3215257d48c1bd3848db809774db, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:29.133780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:29.141978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 864/1722] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:29.143672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:29.158838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.168020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 859/1723] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:28 2022] PUT /identity/v3/projects/f120fb97f91f488bb350faa71094e5e9/users/019c3215257d48c1bd3848db809774db/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:29.169480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:29.207492 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fab89e-1681-4e80-a1f4-f3ad4583f88c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:29.300520 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56064f-1151-46ea-a81f-5747fc611c0c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:29.308170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:29.308946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.309269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.317519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] PATH_INFO: `/v3/users/0e67bd2d2b8c4dde91c157d96ec00d1a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.319229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e67bd2d2b8c4dde91c157d96ec00d1a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:29.322148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:29.324498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcb191-89c9-4258-8216-594f82221898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:29.325192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb191-89c9-4258-8216-594f82221898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.325935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb191-89c9-4258-8216-594f82221898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.325935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcb191-89c9-4258-8216-594f82221898 admin admin] PATH_INFO: `/v3/projects/16b8dd84d08e4bacb20d2b8b5387407b/users/d078d883d5f34714bb233797553a484e/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.328362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcb191-89c9-4258-8216-594f82221898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b8dd84d08e4bacb20d2b8b5387407b, user_id=d078d883d5f34714bb233797553a484e, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:29.331899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcb191-89c9-4258-8216-594f82221898 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:29.374188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 860/1724] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:29 2022] PUT /identity/v3/projects/16b8dd84d08e4bacb20d2b8b5387407b/users/d078d883d5f34714bb233797553a484e/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:29.381239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8440bf-6e77-402b-aa87-151e8cd8c555 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.381534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8440bf-6e77-402b-aa87-151e8cd8c555 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.381818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8440bf-6e77-402b-aa87-151e8cd8c555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.475932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4c8440bf-6e77-402b-aa87-151e8cd8c555 None None] MFA Rules not processed for user `019c3215257d48c1bd3848db809774db`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:29.477363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4c8440bf-6e77-402b-aa87-151e8cd8c555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.502725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:29.614349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e67bd2d2b8c4dde91c157d96ec00d1a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:29.615025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e67bd2d2b8c4dde91c157d96ec00d1a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:29.624230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 861/1725] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:29.626784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.627055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.627327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.660900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] MFA Rules not processed for user `d078d883d5f34714bb233797553a484e`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:29.662376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.704850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:29.706302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e67bd2d2b8c4dde91c157d96ec00d1a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:29.727149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e67bd2d2b8c4dde91c157d96ec00d1a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:29.728738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-143e0bdd-4784-4049-86d4-393f84247f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e67bd2d2b8c4dde91c157d96ec00d1a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:29.749581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 865/1726] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:29 2022] DELETE /identity/v3/users/0e67bd2d2b8c4dde91c157d96ec00d1a => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:29.751069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.751426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.772667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.811989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 862/1727] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:29.812909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:29.818816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.849241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] MFA Rules not processed for user `23d2938afd97422795662ec0581d4fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:29.850898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.968039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 866/1728] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:29.970116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:29.985766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.992459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22bf35-e933-486f-8e33-423a9b9fbfcb None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:30.119383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:30.121205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:30.122089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:30.122433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:30.127625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:30.193406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de697ae-5164-40be-8f7a-af1df3e6b82d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:30.207453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:30.211260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:30.211874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:30.212472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] PATH_INFO: `/v3/projects/480e9449f5e9458eb64fa91701d5be6c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:30.221460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=480e9449f5e9458eb64fa91701d5be6c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:30.224908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:30.284766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:30.287511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:30.318154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 863/1729] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:30.318884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:30.325290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:30.336794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2c3c1-6f8e-48e7-aa18-489e318531a4 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:30.351990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:30.354383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:30.355111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:30.355400 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:30.372663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480e9449f5e9458eb64fa91701d5be6c', 'request_id': 'req-47940cfc-7f9d-48f2-ad65-b199b9e354ed'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:30.387585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480e9449f5e9458eb64fa91701d5be6c', 'request_id': 'req-47940cfc-7f9d-48f2-ad65-b199b9e354ed'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:30.402722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480e9449f5e9458eb64fa91701d5be6c', 'request_id': 'req-47940cfc-7f9d-48f2-ad65-b199b9e354ed'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:30.433440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:30.504136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] The token cache is being invalidate because project 480e9449f5e9458eb64fa91701d5be6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:30.504136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:30.504486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:30.504953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-47940cfc-7f9d-48f2-ad65-b199b9e354ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:30.506478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:30.517861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 867/1730] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:29 2022] DELETE /identity/v3/projects/480e9449f5e9458eb64fa91701d5be6c => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:30.517861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5db84-1115-463c-b80a-789b8334a284 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:30.517861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5db84-1115-463c-b80a-789b8334a284 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:30.517861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5db84-1115-463c-b80a-789b8334a284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:30.525882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 864/1731] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:30.527044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:30.531572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:30.558092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b9c5db84-1115-463c-b80a-789b8334a284 None None] MFA Rules not processed for user `23d2938afd97422795662ec0581d4fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:30.559538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b9c5db84-1115-463c-b80a-789b8334a284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:30.729297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 868/1732] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:30.737450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820b8e-fb94-4629-9572-4a06dde885ec None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:30.742004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820b8e-fb94-4629-9572-4a06dde885ec None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:30.742567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820b8e-fb94-4629-9572-4a06dde885ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:30.746409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef0e8c-419b-4c30-b884-2b76b1e5e210 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:30.800353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4d820b8e-fb94-4629-9572-4a06dde885ec None None] MFA Rules not processed for user `77b0df6c83a84dbcb0fa5802c93dc6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:30.816586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4d820b8e-fb94-4629-9572-4a06dde885ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:30.851025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:30.851755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:30.852044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:30.852321 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:30.861907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:30.879403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:30.881495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:30.894861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 865/1733] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:30.895866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:30.897284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:30.920639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:30.995917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 869/1734] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:30.995917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.007667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.011777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.021206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc98f9b-29f6-4e1b-93a9-27ed40099bf1 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:31.035563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:31.036285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.036595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.036871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] PATH_INFO: `/v3/projects/34cc584a026b4659b34ceecead099673` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.042222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=34cc584a026b4659b34ceecead099673)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:31.047375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:31.074286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:31.081079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.102218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 870/1735] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:31.115622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:31.119388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.277990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34cc584a026b4659b34ceecead099673', 'request_id': 'req-630e30ec-79f0-4313-9485-60947adcccfe'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:31.326039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34cc584a026b4659b34ceecead099673', 'request_id': 'req-630e30ec-79f0-4313-9485-60947adcccfe'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:31.360427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34cc584a026b4659b34ceecead099673', 'request_id': 'req-630e30ec-79f0-4313-9485-60947adcccfe'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:31.424142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd025-8bb3-4b48-8b42-8e80deb66761 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:31.464710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] The token cache is being invalidate because project 34cc584a026b4659b34ceecead099673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:31.465065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:31.466420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-630e30ec-79f0-4313-9485-60947adcccfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:31.468642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 866/1736] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:30 2022] DELETE /identity/v3/projects/34cc584a026b4659b34ceecead099673 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:31.471004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ceff3-2ae2-4242-837e-d4e886d5b0ba None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.471315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ceff3-2ae2-4242-837e-d4e886d5b0ba None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.471601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ceff3-2ae2-4242-837e-d4e886d5b0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.545938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9e5ceff3-2ae2-4242-837e-d4e886d5b0ba None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:31.547204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9e5ceff3-2ae2-4242-837e-d4e886d5b0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.589464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33726d-954d-40fc-8c5d-31af05230744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:31.591499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33726d-954d-40fc-8c5d-31af05230744 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.591804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33726d-954d-40fc-8c5d-31af05230744 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.592084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33726d-954d-40fc-8c5d-31af05230744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.599720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ce33726d-954d-40fc-8c5d-31af05230744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.669020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 867/1737] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:31.670941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ac828-c9bb-48f9-ad8b-f557d8740e17 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.671279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ac828-c9bb-48f9-ad8b-f557d8740e17 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.671564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ac828-c9bb-48f9-ad8b-f557d8740e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.712996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6b7ac828-c9bb-48f9-ad8b-f557d8740e17 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:31.714894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6b7ac828-c9bb-48f9-ad8b-f557d8740e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.758789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 868/1738] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:31.760267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf63b-b6fc-4634-93d4-8e4d4ac8e6d2 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.760484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf63b-b6fc-4634-93d4-8e4d4ac8e6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.760869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbf63b-b6fc-4634-93d4-8e4d4ac8e6d2 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.762058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33726d-954d-40fc-8c5d-31af05230744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:31.767176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 869/1739] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 27 20:13:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:31.767810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbf63b-b6fc-4634-93d4-8e4d4ac8e6d2 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:31.768335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33726d-954d-40fc-8c5d-31af05230744 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:31.787768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbf63b-b6fc-4634-93d4-8e4d4ac8e6d2 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:31.796826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a6dad-4aae-4e01-9c33-9c0211f93c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:31.797463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6dad-4aae-4e01-9c33-9c0211f93c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.797846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6dad-4aae-4e01-9c33-9c0211f93c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.798092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a6dad-4aae-4e01-9c33-9c0211f93c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.798813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a6dad-4aae-4e01-9c33-9c0211f93c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:31.799122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a6dad-4aae-4e01-9c33-9c0211f93c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:31.800784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a6dad-4aae-4e01-9c33-9c0211f93c6b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:31.802453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 871/1740] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:31.807162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.807473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.807759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.822562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 870/1741] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:31.825431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c69c68-7af3-42af-9381-073ade6b9ce1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.825880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c69c68-7af3-42af-9381-073ade6b9ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.826373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c69c68-7af3-42af-9381-073ade6b9ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.832992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:31.834235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.849040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 872/1742] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:31.849040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:31.851909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.867625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-40c69c68-7af3-42af-9381-073ade6b9ce1 None None] MFA Rules not processed for user `54aedfbb64c54c589003b335fde2a17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:31.870221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-40c69c68-7af3-42af-9381-073ade6b9ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.979919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 871/1743] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:13:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:31.981722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c69c68-7af3-42af-9381-073ade6b9ce1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.008370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c69c68-7af3-42af-9381-073ade6b9ce1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.019962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.020933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.022259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.023221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.025698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.030206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:32.051570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d3e00-fd40-4a44-9103-b942c359b4e4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.062887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.063387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.070979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.070979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.072158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:32.151114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:32.164805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 872/1744] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:31 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:32.166414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.220647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.222097 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:32.249585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 873/1745] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:32.250470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.252247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:32.337662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8021d62-05fd-4428-812d-afa2decff6ef admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.354908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.357927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.358217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.359652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.363098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:32.363417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.371148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:32.399695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 873/1746] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:32.402870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.411301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:32.444004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0759-fb5a-4ae4-b748-4fcb1da8da44 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.451422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:32.583403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae26d6a-7a10-4f23-a915-88ea8c2a3bbd admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.592333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c3215257d48c1bd3848db809774db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f120fb97f91f488bb350faa71094e5e9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.592333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.592668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.593180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.593890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:32.594204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.595002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.595658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.596948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.596948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.601825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:32.616749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.620138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:32.627084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 874/1747] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:32.629437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.661115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bac235-9109-4a50-8d52-a919c3beca93 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.662708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 874/1748] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:32.665133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.668768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.669833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.670284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.670793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.672434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.675761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:32.705825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b32aed-c358-44dd-800e-d686d13ed927 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.713831 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.715086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.715566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.717055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.717887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.720954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:32.796665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 875/1749] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:32 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:32.799057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.849234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894ef3-f9f2-4aac-9d2e-44e5f9e43878 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:32.873419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 875/1750] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:32 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:32.874490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:32.886491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c3215257d48c1bd3848db809774db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f120fb97f91f488bb350faa71094e5e9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:32.893187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:32.893187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:32.893909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:32.896874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:32.908565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.124011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 876/1751] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:32 2022] POST /identity/v3/projects => generated 354 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:33.131616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:33.159504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37331dbb-20bc-4d59-a9bf-7b22c542d89a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.399022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.399718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.400234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.400329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.405824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.407822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.424052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 876/1752] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 547 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:33.424052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:33.452057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b558-fa06-40ac-b286-d586765c0567 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.474234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.474865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.475155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.475430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.480918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.492497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.649073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 877/1753] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:33 2022] POST /identity/v3/users => generated 293 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:33.650043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:33.669418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881c63-94a2-493e-81cd-b1c654ef0986 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.697176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f80aaa-2864-46ba-9e38-88a587134223 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.703995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c3215257d48c1bd3848db809774db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f120fb97f91f488bb350faa71094e5e9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.704891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.705281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.707346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.708167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:33.708564 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.711106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.728554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 877/1754] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:33.728554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:33.730303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.730946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.731231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.731507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.732221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.740016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91825a02-8151-4c1c-a405-241068572ee2 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.741358 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.752593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.753228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.753504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.753779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.754485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.756090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.767537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 878/1755] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:33.769070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:33.785795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 878/1756] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:33.791181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6822a6-f34c-46dc-b031-ddf795ec5ace admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.791536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:33.799523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.800338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.800772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.801153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] PATH_INFO: `/v3/projects/485bfc95c88041abac2dc6a077c968e9/users/f1a68201089f45b7873180145c4cd583/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.813839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74889-8b6c-4e24-842f-b0fea5eaeb23 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.827972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.828975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.829270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.829593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.832241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.834607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.835874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] RBAC: Authorizing `identity:create_grant(project_id=485bfc95c88041abac2dc6a077c968e9, user_id=f1a68201089f45b7873180145c4cd583, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.843095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.863516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 879/1757] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:33.866464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:33.892035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde959f-f8f5-4523-a4fe-a62631d301fa admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:33.911018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '019c3215257d48c1bd3848db809774db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f120fb97f91f488bb350faa71094e5e9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:33.911833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:33.911833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:33.912868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] PATH_INFO: `/v3/projects/6f53a8beb05a49cc8a95cc3145d7d35d` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:33.913673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6f53a8beb05a49cc8a95cc3145d7d35d)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:33.918373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:33.978227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 879/1758] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:33 2022] PUT /identity/v3/projects/485bfc95c88041abac2dc6a077c968e9/users/f1a68201089f45b7873180145c4cd583/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:33.978883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:34.068566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f53a8beb05a49cc8a95cc3145d7d35d', 'request_id': 'req-fd92d811-b954-48c1-8e90-4f05f0f419c0'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:34.082347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f53a8beb05a49cc8a95cc3145d7d35d', 'request_id': 'req-fd92d811-b954-48c1-8e90-4f05f0f419c0'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:34.086162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:34.097066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f53a8beb05a49cc8a95cc3145d7d35d', 'request_id': 'req-fd92d811-b954-48c1-8e90-4f05f0f419c0'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:34.105142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0316bb4-58be-4031-8293-aadb07ee651d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:34.175469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef788a-3576-419a-bfcf-0fe1a8e6f760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:34.176307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef788a-3576-419a-bfcf-0fe1a8e6f760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:34.176725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef788a-3576-419a-bfcf-0fe1a8e6f760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:34.177098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef788a-3576-419a-bfcf-0fe1a8e6f760 admin admin] PATH_INFO: `/v3/projects/57d005c65dfb4d97b5f668ab37318f69/users/724c74af403e40e4bfe0877e3403ef8a/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:34.195851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef788a-3576-419a-bfcf-0fe1a8e6f760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d005c65dfb4d97b5f668ab37318f69, user_id=724c74af403e40e4bfe0877e3403ef8a, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:34.199237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef788a-3576-419a-bfcf-0fe1a8e6f760 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:34.201862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] The token cache is being invalidate because project 6f53a8beb05a49cc8a95cc3145d7d35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:34.202363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:34.204725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fd92d811-b954-48c1-8e90-4f05f0f419c0 tempest-VolumeQuotaClassesTest-1976636648 tempest-VolumeQuotaClassesTest-1976636648-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:34.209137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 880/1759] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:33 2022] DELETE /identity/v3/projects/6f53a8beb05a49cc8a95cc3145d7d35d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:34.214046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:34.214672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:34.215093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:34.276525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 880/1760] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:33 2022] PUT /identity/v3/projects/57d005c65dfb4d97b5f668ab37318f69/users/724c74af403e40e4bfe0877e3403ef8a/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:34.278114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662b20-caf4-4866-97d5-635587f0be7a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:34.280665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662b20-caf4-4866-97d5-635587f0be7a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:34.281108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59662b20-caf4-4866-97d5-635587f0be7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:34.299608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] MFA Rules not processed for user `f1a68201089f45b7873180145c4cd583`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:34.301151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:34.319132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-59662b20-caf4-4866-97d5-635587f0be7a None None] MFA Rules not processed for user `6cc262a3c3314be8a6ee620ebfa6fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:34.333268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-59662b20-caf4-4866-97d5-635587f0be7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:34.469655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:34.485001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 881/1761] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:34.486931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:34.487458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:34.487978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:34.516718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 881/1762] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:34.521264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:34.523417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:34.524035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] MFA Rules not processed for user `724c74af403e40e4bfe0877e3403ef8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:34.526449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:34.638315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 882/1763] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:34.643100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:34.644841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:34.698525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82870de-5a8c-4fc0-a099-08ee27a99762 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:34.703984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:34.704799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:34.705136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:34.705471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:34.706271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:34.708861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:34.791729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0964ba-cd14-4f86-a0d3-e25a31999217 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:34.797757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:34.806786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:34.808397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:34.808889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:34.810360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:34.819221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:34.837310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 882/1764] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:34 2022] POST /identity/v3/projects => generated 363 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:34.843740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:34.973820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 883/1765] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:34 2022] POST /identity/v3/projects => generated 385 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:34.978426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:35.066184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513924c8-54d7-467c-8b8f-db768c3e5bb4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:35.222094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:35.223418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:35.223844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:35.224310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:35.225360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:35.228863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:35.309670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73f1bc-8e59-4639-bed0-6cf2c2c98872 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:35.316109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:35.317173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:35.317910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:35.318593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:35.319433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:35.338242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:35.377280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 883/1766] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:34 2022] POST /identity/v3/users => generated 288 bytes in 534 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:35.378552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:35.381957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:35.397572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea88af-7030-41c8-ba4c-2faa8149c302 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:35.410824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:35.411545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:35.411833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:35.412111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] PATH_INFO: `/v3/users/a0caf208424b463bb1aefb69311dd482` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:35.414252 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0caf208424b463bb1aefb69311dd482)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:35.423786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 884/1767] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:34 2022] POST /identity/v3/users => generated 299 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:35.425383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:35.429744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:35.447661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c1990-b85f-4306-84c8-391702f4bb69 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:35.472268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:35.478710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:35.479211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:35.488254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:35.490625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:35.499039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:35.524311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 885/1768] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:35.526051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:35.547534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b87b3-1334-4384-9d1d-64b297cba67a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:35.553635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:35.554597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:35.555076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:35.555497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:35.557095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:35.559093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:35.572330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 886/1769] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:35.574008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:35.588123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0c9cd-44e6-470f-bd98-f577c55cf199 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:35.593911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:35.594651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:35.594938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:35.595216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] PATH_INFO: `/v3/projects/f398f093efa74c73b913169c1f849d7f/users/76a3486a511d454c9f4ef262140f0696/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:35.628517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f398f093efa74c73b913169c1f849d7f, user_id=76a3486a511d454c9f4ef262140f0696, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:35.631551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:35.724240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 887/1770] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:35 2022] PUT /identity/v3/projects/f398f093efa74c73b913169c1f849d7f/users/76a3486a511d454c9f4ef262140f0696/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:35.725321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:35.759766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0caf208424b463bb1aefb69311dd482'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:35.772013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0caf208424b463bb1aefb69311dd482'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:35.863238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f5e76-c58f-4eed-8551-46e88b1bd9c2 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:35.880729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17eecd-c18d-46bc-8124-3130882521bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:35.880729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17eecd-c18d-46bc-8124-3130882521bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:35.880729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17eecd-c18d-46bc-8124-3130882521bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:35.880729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17eecd-c18d-46bc-8124-3130882521bc admin admin] PATH_INFO: `/v3/projects/fdc36078225c4a02a793286d76e1bad9/users/c90ac84653234ceabdcdef70a290508e/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:35.896417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0caf208424b463bb1aefb69311dd482'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:35.916287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0caf208424b463bb1aefb69311dd482'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:35.922124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17eecd-c18d-46bc-8124-3130882521bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc36078225c4a02a793286d76e1bad9, user_id=c90ac84653234ceabdcdef70a290508e, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:35.925064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0caf208424b463bb1aefb69311dd482'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:35.926207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17eecd-c18d-46bc-8124-3130882521bc admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:35.933529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 884/1771] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:35 2022] DELETE /identity/v3/users/a0caf208424b463bb1aefb69311dd482 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:35.935915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:35.944660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:36.043901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 888/1772] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:35 2022] PUT /identity/v3/projects/fdc36078225c4a02a793286d76e1bad9/users/c90ac84653234ceabdcdef70a290508e/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:36.045284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.045569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.050142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.052812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95167e40-05cd-4c4c-bf02-c9883437ae45 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:36.069900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:36.099112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:36.100409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:36.224806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 889/1773] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:36.229203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:36.231040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:36.259900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:36.260729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.261014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.261288 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.262130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:36.264614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:36.272362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb35c4-44f1-48b8-856f-0a17940cb735 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:36.275998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:36.283207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 885/1774] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:36.284178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:36.286013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:36.291568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:36.294042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.294793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.295079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.295804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:36.300503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:36.318222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 890/1775] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:36.319292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:36.342801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273199a-0d73-4c3a-989e-c4d89bc48ca7 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:36.349984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:36.350890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.351180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.351461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] PATH_INFO: `/v3/projects/f398f093efa74c73b913169c1f849d7f/users/76a3486a511d454c9f4ef262140f0696/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.354164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f398f093efa74c73b913169c1f849d7f, user_id=76a3486a511d454c9f4ef262140f0696, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:36.357238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:36.370245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a6f01-a470-4d67-a909-bf91b953bba0 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:36.374095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 891/1776] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:36 2022] PUT /identity/v3/projects/f398f093efa74c73b913169c1f849d7f/users/76a3486a511d454c9f4ef262140f0696/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:36.374761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:36.453055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d330a6-6c02-4cfd-be42-ce49ade69719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:36.454881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9510fd3-557a-45fb-9b8a-20aa950b77e7 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:36.456318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d330a6-6c02-4cfd-be42-ce49ade69719 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.456634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d330a6-6c02-4cfd-be42-ce49ade69719 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.456917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d330a6-6c02-4cfd-be42-ce49ade69719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.461195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48efaa1-f24c-473a-b837-0922c562cb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:36.462147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48efaa1-f24c-473a-b837-0922c562cb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.464297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48efaa1-f24c-473a-b837-0922c562cb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.464297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48efaa1-f24c-473a-b837-0922c562cb86 admin admin] PATH_INFO: `/v3/projects/fdc36078225c4a02a793286d76e1bad9/users/c90ac84653234ceabdcdef70a290508e/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.465885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48efaa1-f24c-473a-b837-0922c562cb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc36078225c4a02a793286d76e1bad9, user_id=c90ac84653234ceabdcdef70a290508e, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:36.468638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48efaa1-f24c-473a-b837-0922c562cb86 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:36.470663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d330a6-6c02-4cfd-be42-ce49ade69719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:36.472214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d330a6-6c02-4cfd-be42-ce49ade69719 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:36.479676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 892/1777] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:36 2022] PUT /identity/v3/projects/fdc36078225c4a02a793286d76e1bad9/users/c90ac84653234ceabdcdef70a290508e/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:36.481026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.481315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.481594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.482589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 886/1778] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:36.489921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.490283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.492656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.522951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] MFA Rules not processed for user `c90ac84653234ceabdcdef70a290508e`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:36.523143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] MFA Rules not processed for user `76a3486a511d454c9f4ef262140f0696`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:36.524498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:36.525858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:36.614704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 893/1779] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:36 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:36.616553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 887/1780] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:36.713805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:36.715272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:36.802495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167be5f-762e-47e3-bd5c-e61223e85d96 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:36.823610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:36.824443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:36.824829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:36.825209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:36.829561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:36.847133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:36.849242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:36.865978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 894/1781] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:36.963910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:36.975871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:37.002084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606a29a-1de6-4279-b0af-dd0dc164079a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:37.007182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:37.008178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:37.008722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:37.009107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:37.059172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:37.061117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:37.084082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:37.085667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:37.087991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 888/1782] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:37.161895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:37.297383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4d6f3-6354-470d-a428-c9bee4549a3b service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:37.308963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:37.308963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:37.308963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:37.308963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] PATH_INFO: `/v3/projects/4fcdb4d877064f7dbfd5c2dfb813d3d4` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:37.309796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fcdb4d877064f7dbfd5c2dfb813d3d4)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:37.310737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:37.375985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097de7d-2b33-4f98-978c-483fad0aeadb service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:37.389371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:37.390481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:37.391010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:37.391511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] PATH_INFO: `/v3/users/724c74af403e40e4bfe0877e3403ef8a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:37.393091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724c74af403e40e4bfe0877e3403ef8a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:37.395283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:37.587872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fcdb4d877064f7dbfd5c2dfb813d3d4', 'request_id': 'req-6d2d4d30-2561-435d-9624-2bd57ca64701'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.602007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fcdb4d877064f7dbfd5c2dfb813d3d4', 'request_id': 'req-6d2d4d30-2561-435d-9624-2bd57ca64701'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.625102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fcdb4d877064f7dbfd5c2dfb813d3d4', 'request_id': 'req-6d2d4d30-2561-435d-9624-2bd57ca64701'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.708366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724c74af403e40e4bfe0877e3403ef8a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.709408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724c74af403e40e4bfe0877e3403ef8a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.754336 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] The token cache is being invalidate because project 4fcdb4d877064f7dbfd5c2dfb813d3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:37.754617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.755485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.762075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 895/1783] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:37 2022] DELETE /identity/v3/projects/4fcdb4d877064f7dbfd5c2dfb813d3d4 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:37.762968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:37.764830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:37.834362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d4d30-2561-435d-9624-2bd57ca64701 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:37.842113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724c74af403e40e4bfe0877e3403ef8a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.843193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:37.843966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:37.844442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:37.849595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] PATH_INFO: `/v3/users/019c3215257d48c1bd3848db809774db` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:37.849595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=019c3215257d48c1bd3848db809774db)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:37.849595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:37.858660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724c74af403e40e4bfe0877e3403ef8a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.866647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-06e97d79-d0ff-4529-a26a-0f56f1190061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724c74af403e40e4bfe0877e3403ef8a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:37.875308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 889/1784] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:37 2022] DELETE /identity/v3/users/724c74af403e40e4bfe0877e3403ef8a => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:37.881665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:37.882204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:37.882548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:37.903789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:37.905043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:37.984519 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 890/1785] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:38.147388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019c3215257d48c1bd3848db809774db'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:38.147388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019c3215257d48c1bd3848db809774db'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:38.234702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019c3215257d48c1bd3848db809774db'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:38.237894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:38.240415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:38.277110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019c3215257d48c1bd3848db809774db'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:38.278595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-007c8f9a-1882-4645-8825-4d5aed90e087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019c3215257d48c1bd3848db809774db'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:38.314417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 896/1786] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:37 2022] DELETE /identity/v3/users/019c3215257d48c1bd3848db809774db => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:38.325005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:38.325333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:38.325614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:38.410441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fbf6d-4e92-481b-b288-17a21f9e32b0 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:38.421119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:38.422682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:38.520742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:38.521761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:38.522214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:38.522648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:38.527521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:38.534279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:38.560354 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 897/1787] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:13:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:38.677810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:38.678391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:38.679959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:38.682396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:38.706216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 891/1788] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:38.723223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdff909-e28c-434e-a48c-cba512d95bd4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:38.737714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:38.739870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:38.740160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:38.740463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] PATH_INFO: `/v3/users/f1a68201089f45b7873180145c4cd583` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:38.745156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a68201089f45b7873180145c4cd583)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:38.750521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:38.811578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:38.910969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:38.912905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:38.961959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a68201089f45b7873180145c4cd583'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:38.962744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a68201089f45b7873180145c4cd583'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.001546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:39.017669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a68201089f45b7873180145c4cd583'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.036094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a68201089f45b7873180145c4cd583'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.039248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9b99f5c2-0139-4af9-bd21-acf78cb2cf14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a68201089f45b7873180145c4cd583'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.059554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 898/1789] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:38 2022] DELETE /identity/v3/users/f1a68201089f45b7873180145c4cd583 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:39.065149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:39.065937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:39.066249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:39.088798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:39.090163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:39.117770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fd49b-e4ea-4120-a553-70823ddb631e service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:39.186011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 899/1790] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:39.197831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:39.199550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:39.212143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3b536-3cdd-49e9-b739-8bc00df5369b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:39.212830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3b536-3cdd-49e9-b739-8bc00df5369b service swift] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:39.213116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3b536-3cdd-49e9-b739-8bc00df5369b service swift] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:39.213741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3b536-3cdd-49e9-b739-8bc00df5369b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:39.215733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f0a3b536-3cdd-49e9-b739-8bc00df5369b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:39.286805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a36005-3774-406a-8dbf-15c7848b0799 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:39.293018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:39.293866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:39.294645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:39.295444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:39.311031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:39.313209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:39.344310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 900/1791] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:39.346022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:39.349386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:39.358412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3b536-3cdd-49e9-b739-8bc00df5369b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:39.360903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3b536-3cdd-49e9-b739-8bc00df5369b service swift] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:39.398063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8240c6-9447-4251-8ab3-56d8ec326995 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:39.399758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 892/1792] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:13:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:13:39.434858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:39.435622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:39.435907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:39.436199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] PATH_INFO: `/v3/projects/f120fb97f91f488bb350faa71094e5e9` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:39.441780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f120fb97f91f488bb350faa71094e5e9)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:39.449515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:39.647407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f120fb97f91f488bb350faa71094e5e9', 'request_id': 'req-3c75be3e-86c7-4571-b2d3-ac203e189d4f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.664635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f120fb97f91f488bb350faa71094e5e9', 'request_id': 'req-3c75be3e-86c7-4571-b2d3-ac203e189d4f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.677850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f120fb97f91f488bb350faa71094e5e9', 'request_id': 'req-3c75be3e-86c7-4571-b2d3-ac203e189d4f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.774513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] The token cache is being invalidate because project f120fb97f91f488bb350faa71094e5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:39.774923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.776123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3c75be3e-86c7-4571-b2d3-ac203e189d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:39.785992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 901/1793] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:39 2022] DELETE /identity/v3/projects/f120fb97f91f488bb350faa71094e5e9 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:39.788206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53690dd-a2b5-4fb9-a25c-76ea62418f99 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:39.789118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53690dd-a2b5-4fb9-a25c-76ea62418f99 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:39.789627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53690dd-a2b5-4fb9-a25c-76ea62418f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:39.813256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-e53690dd-a2b5-4fb9-a25c-76ea62418f99 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:39.814854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e53690dd-a2b5-4fb9-a25c-76ea62418f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:39.909054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 893/1794] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:39.910473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810bb22-2aff-42c0-b9b1-37264de0101c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:39.910768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810bb22-2aff-42c0-b9b1-37264de0101c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:39.911483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810bb22-2aff-42c0-b9b1-37264de0101c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:39.960500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a810bb22-2aff-42c0-b9b1-37264de0101c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:39.962476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a810bb22-2aff-42c0-b9b1-37264de0101c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:39.981664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 902/1795] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:39.981849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53690dd-a2b5-4fb9-a25c-76ea62418f99 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:40.002435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53690dd-a2b5-4fb9-a25c-76ea62418f99 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:40.009898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:40.010837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:40.011877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:40.012346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:40.013548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:40.014069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:40.016324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:40.048161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 894/1796] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:40.052746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810bb22-2aff-42c0-b9b1-37264de0101c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:40.063703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:40.067081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810bb22-2aff-42c0-b9b1-37264de0101c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:40.081805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:40.082464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:40.090345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:40.095740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:40.098028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:40.099966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:40.198388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:40.206928 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:40.250483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340ec19-d0e2-43f9-b654-b85b8c122aa2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:40.270129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:40.270869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:40.271234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:40.271590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] PATH_INFO: `/v3/projects/57d005c65dfb4d97b5f668ab37318f69` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:40.273687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d005c65dfb4d97b5f668ab37318f69)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:40.275777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:40.391836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 903/1797] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:40 2022] POST /identity/v3/projects => generated 379 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:40.395748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:40.641526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ac4fc-7ec5-44f8-bcd5-9fdb7894803e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:40.641526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:40.641526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:40.641526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:40.642552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:40.642552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:40.642552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:40.758673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d005c65dfb4d97b5f668ab37318f69', 'request_id': 'req-a955f722-c841-4958-be9c-8133522707d1'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:40.775795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d005c65dfb4d97b5f668ab37318f69', 'request_id': 'req-a955f722-c841-4958-be9c-8133522707d1'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:40.775795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d005c65dfb4d97b5f668ab37318f69', 'request_id': 'req-a955f722-c841-4958-be9c-8133522707d1'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:40.796051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 904/1798] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:40 2022] POST /identity/v3/users => generated 290 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:40.806469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:40.806469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:40.812943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a5f5b-9a64-4ff5-9784-aae3cd813cfb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:40.886123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:40.886740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:40.887023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:40.887298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] PATH_INFO: `/v3/projects/485bfc95c88041abac2dc6a077c968e9` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:40.888638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=485bfc95c88041abac2dc6a077c968e9)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:40.892606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:40.993670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] The token cache is being invalidate because project 57d005c65dfb4d97b5f668ab37318f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:40.994474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:40.995098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:40.996828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 895/1799] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:40 2022] DELETE /identity/v3/projects/57d005c65dfb4d97b5f668ab37318f69 => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:40.997522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:41.000732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:41.015028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955f722-c841-4958-be9c-8133522707d1 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:41.023496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:41.023496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:41.023496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:41.023496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:41.024022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:41.024022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:41.039268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 896/1800] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:41.039268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:41.042686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:41.060498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b405a2-67cb-4417-9fe1-f842959946e4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:41.072213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485bfc95c88041abac2dc6a077c968e9', 'request_id': 'req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.084005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485bfc95c88041abac2dc6a077c968e9', 'request_id': 'req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.098102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:41.099793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485bfc95c88041abac2dc6a077c968e9', 'request_id': 'req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.171453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:41.172130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:41.172442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:41.172724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] PATH_INFO: `/v3/users/c90ac84653234ceabdcdef70a290508e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:41.174350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90ac84653234ceabdcdef70a290508e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:41.176844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:41.198495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] The token cache is being invalidate because project 485bfc95c88041abac2dc6a077c968e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:41.200448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.200448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.201732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 905/1801] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:40 2022] DELETE /identity/v3/projects/485bfc95c88041abac2dc6a077c968e9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:41.202801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:41.206579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:41.285213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b1eae-bd00-45c6-8b85-5b16fb0dcfe1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:41.299600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:41.301089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:41.301499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:41.301877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:41.302983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:41.305228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:41.333849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 906/1802] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:41.341835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:41.347260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:41.377296 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b32c2d-36a0-4e6e-96af-35936474581d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:41.395983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:41.396688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:41.396973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:41.397258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] PATH_INFO: `/v3/users/76a3486a511d454c9f4ef262140f0696` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:41.400536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a3486a511d454c9f4ef262140f0696)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:41.402475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:41.498678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90ac84653234ceabdcdef70a290508e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.499443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90ac84653234ceabdcdef70a290508e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.600469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90ac84653234ceabdcdef70a290508e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.614229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90ac84653234ceabdcdef70a290508e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.622306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90ac84653234ceabdcdef70a290508e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.628310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 897/1803] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:41 2022] DELETE /identity/v3/users/c90ac84653234ceabdcdef70a290508e => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:41.631918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:41.633308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:41.646034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:41.667041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a3486a511d454c9f4ef262140f0696'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.668159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a3486a511d454c9f4ef262140f0696'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.716824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318c54-ed09-4117-81ca-90df5d117f07 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:41.721691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:41.722380 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:41.722687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:41.722964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] PATH_INFO: `/v3/projects/62a5eed2e9c441f9835b1cc4ab0bb74a/users/d28d55cc6b1548a9b87fac2f0d509f13/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:41.740072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a5eed2e9c441f9835b1cc4ab0bb74a, user_id=d28d55cc6b1548a9b87fac2f0d509f13, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:41.742842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:41.770964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a3486a511d454c9f4ef262140f0696'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.784201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a3486a511d454c9f4ef262140f0696'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.784914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5572653c-b9b3-4a6a-a3da-a6ac6163d711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a3486a511d454c9f4ef262140f0696'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:41.801599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 907/1804] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:41 2022] DELETE /identity/v3/users/76a3486a511d454c9f4ef262140f0696 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:41.851919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 898/1805] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:41 2022] PUT /identity/v3/projects/62a5eed2e9c441f9835b1cc4ab0bb74a/users/d28d55cc6b1548a9b87fac2f0d509f13/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:41.864508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a69d9-574f-4a37-a40a-c06a5233f0de None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:41.864829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a69d9-574f-4a37-a40a-c06a5233f0de None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:41.865105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a69d9-574f-4a37-a40a-c06a5233f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:41.897603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6f6a69d9-574f-4a37-a40a-c06a5233f0de None None] MFA Rules not processed for user `d28d55cc6b1548a9b87fac2f0d509f13`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:41.898811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6f6a69d9-574f-4a37-a40a-c06a5233f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:41.966116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 908/1806] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:41.970758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:41.972526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:42.038548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda72ac6-3116-4564-8674-fec40373d3d4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:42.043819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:42.044571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.044790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.045068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.045764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:42.047672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:42.202995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 899/1807] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:41 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:42.208466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a69d9-574f-4a37-a40a-c06a5233f0de None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:42.220828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a69d9-574f-4a37-a40a-c06a5233f0de None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:42.225843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:42.279090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:42.283128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:42.321819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:42.322847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.323283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.323687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.324663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:42.328958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:42.405814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f03101-4760-462d-9bca-f91d4809e5b8 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:42.411271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:42.412087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.412499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.412872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] PATH_INFO: `/v3/projects/fdc36078225c4a02a793286d76e1bad9` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.414340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc36078225c4a02a793286d76e1bad9)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:42.417195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:42.440485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:42.463993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 909/1808] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:42 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:42.463993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:42.471157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:42.490320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d784d51-c363-4f14-bdae-2ae504df4890 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:42.522056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ea409-406a-4827-b538-81895091209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:42.522842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea409-406a-4827-b538-81895091209d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.523208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea409-406a-4827-b538-81895091209d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.523561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea409-406a-4827-b538-81895091209d admin admin] PATH_INFO: `/v3/projects/f398f093efa74c73b913169c1f849d7f` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.525446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ea409-406a-4827-b538-81895091209d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f398f093efa74c73b913169c1f849d7f)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:42.527491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ea409-406a-4827-b538-81895091209d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:42.556168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc36078225c4a02a793286d76e1bad9', 'request_id': 'req-204bb34c-a345-47e8-991e-7e1399f26fae'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.566626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc36078225c4a02a793286d76e1bad9', 'request_id': 'req-204bb34c-a345-47e8-991e-7e1399f26fae'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.572998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc36078225c4a02a793286d76e1bad9', 'request_id': 'req-204bb34c-a345-47e8-991e-7e1399f26fae'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.694211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] The token cache is being invalidate because project fdc36078225c4a02a793286d76e1bad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:42.694508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.695573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.697751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 900/1809] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:42 2022] DELETE /identity/v3/projects/fdc36078225c4a02a793286d76e1bad9 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:42.698277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:42.699914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:42.710862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-301ea409-406a-4827-b538-81895091209d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f398f093efa74c73b913169c1f849d7f', 'request_id': 'req-301ea409-406a-4827-b538-81895091209d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.724066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-301ea409-406a-4827-b538-81895091209d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f398f093efa74c73b913169c1f849d7f', 'request_id': 'req-301ea409-406a-4827-b538-81895091209d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.732798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-301ea409-406a-4827-b538-81895091209d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f398f093efa74c73b913169c1f849d7f', 'request_id': 'req-301ea409-406a-4827-b538-81895091209d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.812734 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-301ea409-406a-4827-b538-81895091209d admin admin] The token cache is being invalidate because project f398f093efa74c73b913169c1f849d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:42.813029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-301ea409-406a-4827-b538-81895091209d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.813950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-301ea409-406a-4827-b538-81895091209d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:42.820601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 910/1810] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:42 2022] DELETE /identity/v3/projects/f398f093efa74c73b913169c1f849d7f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:42.820601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e752-c691-4eb6-8dd0-76db87469830 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.820601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e752-c691-4eb6-8dd0-76db87469830 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.820601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e752-c691-4eb6-8dd0-76db87469830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.821036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb34c-a345-47e8-991e-7e1399f26fae admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:42.843538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4885e752-c691-4eb6-8dd0-76db87469830 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:42.846422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4885e752-c691-4eb6-8dd0-76db87469830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:42.922512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 911/1811] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:42.924141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.924497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.924777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.930653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71aa637-516f-4f6b-80f3-f7c41a0d681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:42.932862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71aa637-516f-4f6b-80f3-f7c41a0d681d admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.933156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71aa637-516f-4f6b-80f3-f7c41a0d681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.933538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71aa637-516f-4f6b-80f3-f7c41a0d681d admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.934163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71aa637-516f-4f6b-80f3-f7c41a0d681d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:42.937585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71aa637-516f-4f6b-80f3-f7c41a0d681d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:42.953208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 901/1812] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:42.954643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d123-6999-4a9d-83a2-a490f65fbd2b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:42.954938 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d123-6999-4a9d-83a2-a490f65fbd2b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:42.955383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449d123-6999-4a9d-83a2-a490f65fbd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:42.969451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:42.972232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:42.983934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:42.986455 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a449d123-6999-4a9d-83a2-a490f65fbd2b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:42.994503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a449d123-6999-4a9d-83a2-a490f65fbd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:42.995511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 912/1813] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:42.996860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:42.999018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:43.018225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 902/1814] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:43.021364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854fb7-8ad2-44f2-9c21-17304e0ceeb1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.028660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9ee9b-3460-44b1-a1df-3b9449dd6da1 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.028943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854fb7-8ad2-44f2-9c21-17304e0ceeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.028943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00854fb7-8ad2-44f2-9c21-17304e0ceeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.029088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a0e9b-d49a-4dc3-ac96-a384e3379b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.029088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a0e9b-d49a-4dc3-ac96-a384e3379b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.029088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a0e9b-d49a-4dc3-ac96-a384e3379b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.029088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a0e9b-d49a-4dc3-ac96-a384e3379b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.029372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a0e9b-d49a-4dc3-ac96-a384e3379b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.030469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a0e9b-d49a-4dc3-ac96-a384e3379b2d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.047834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 913/1815] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:43.050033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece4c91-d4a4-4fd2-b339-7ffd0fc714be None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.050373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece4c91-d4a4-4fd2-b339-7ffd0fc714be None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.057501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece4c91-d4a4-4fd2-b339-7ffd0fc714be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.057647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-00854fb7-8ad2-44f2-9c21-17304e0ceeb1 None None] MFA Rules not processed for user `54aedfbb64c54c589003b335fde2a17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:43.057647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-00854fb7-8ad2-44f2-9c21-17304e0ceeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:43.088780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5ece4c91-d4a4-4fd2-b339-7ffd0fc714be None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:43.091804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5ece4c91-d4a4-4fd2-b339-7ffd0fc714be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:43.099836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 914/1816] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:43.104311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece4c91-d4a4-4fd2-b339-7ffd0fc714be None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.112594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece4c91-d4a4-4fd2-b339-7ffd0fc714be None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.117950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.122423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.122423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.122423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.122423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:43.122423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.122423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.138259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 915/1817] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:43.139893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.150234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 903/1818] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:43.151409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854fb7-8ad2-44f2-9c21-17304e0ceeb1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.153785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851e8e4-b015-45c2-b94b-2de78b10478f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.166237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.167142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.168565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00854fb7-8ad2-44f2-9c21-17304e0ceeb1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.168982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.169373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] PATH_INFO: `/v3/projects/4c9e9ef506404ea781887ad7507cb711/users/8819a7cea22b48b88177c2e6e8fd0546/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.182260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.183184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.183610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.183894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.184706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:43.188190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.188190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.192650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9e9ef506404ea781887ad7507cb711, user_id=8819a7cea22b48b88177c2e6e8fd0546, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.195888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.205912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 904/1819] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:43.208575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.217439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c3628-069c-40a6-a8b6-7be8cfa9db3e admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.222188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e2496-db60-4ce8-8ebd-a0a495884fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.222915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e2496-db60-4ce8-8ebd-a0a495884fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.223197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e2496-db60-4ce8-8ebd-a0a495884fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.223521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e2496-db60-4ce8-8ebd-a0a495884fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.224188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e2496-db60-4ce8-8ebd-a0a495884fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.226270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e2496-db60-4ce8-8ebd-a0a495884fa5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.303643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 916/1820] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:43 2022] PUT /identity/v3/projects/4c9e9ef506404ea781887ad7507cb711/users/8819a7cea22b48b88177c2e6e8fd0546/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:43.304615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.337774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 905/1821] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:43.339285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d0b4-f110-43e6-9593-c31ace169780 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.339616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d0b4-f110-43e6-9593-c31ace169780 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.340040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d0b4-f110-43e6-9593-c31ace169780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.389379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a567d0b4-f110-43e6-9593-c31ace169780 None None] MFA Rules not processed for user `8819a7cea22b48b88177c2e6e8fd0546`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:43.391731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a567d0b4-f110-43e6-9593-c31ace169780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:43.414419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:43.419614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f606bf-7295-487c-926d-abc75e7df572 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.503385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.504710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.505287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.509468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.509468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.510321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.515373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 906/1822] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:43.516570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d0b4-f110-43e6-9593-c31ace169780 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.535483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d0b4-f110-43e6-9593-c31ace169780 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.544680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.545249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.545713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.545999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.546918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.549309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.659636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 917/1823] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/projects => generated 367 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:43.664257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.671233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 907/1824] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:43.671233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.677775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:43.785352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac83485-4a69-4934-bde7-d8854504c97e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.792118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.793191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.793847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.794295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.795145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.797272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.880958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e6d86-4102-4649-9eba-b7d4e113915f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:43.919893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 918/1825] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/projects => generated 377 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:43.921272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:43.955584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:43.956027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:43.960988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:43.960988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:43.970805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:43.973089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:43.988000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 908/1826] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:43.988725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.014126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b557e-dd92-41af-a787-656bb1ab4cfe admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.016294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e234cbc-fa7c-43c0-a500-00aa0122d85a service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.025785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.028448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.028742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.029021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.030053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.031059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.031339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.031614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.032360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.032963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.036413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.038223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.062915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 909/1827] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:44.071635 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.071635 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:44.128553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:44.147352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 919/1828] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:43 2022] POST /identity/v3/users => generated 284 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:44.148242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.152915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:44.206220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:44.207482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2705195-1591-463c-82ef-0dad5735f482 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.212669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.213298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.213578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.213851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.230888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.232534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.265521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 910/1829] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:44.272783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.281529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebab25-ac11-4ae6-8cb1-bae6174400eb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.291667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.292393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.294913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.295268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.306209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.307748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.309262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a124b-0794-46df-8bc8-4e0a49c74e96 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.324007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 920/1830] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:44.328773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.338378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12261230-c832-4648-8ee4-07d0487725bf service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.342688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.343336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.343641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.343761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.343902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.344613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.344846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.345133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.345424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.346275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.346567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.348141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.366574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 921/1831] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:44.366574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.367361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:44.385632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217c7a4-1445-4942-bdd2-628175f4a9df admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.394186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.395299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.396197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.398390 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] PATH_INFO: `/v3/users/2b2516350efd483faa420d25463ce320` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.400667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2516350efd483faa420d25463ce320)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.403506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.473269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 911/1832] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:44 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:44.474224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.498215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274ba9b-0c93-4bbe-9131-b31e1039931f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.516730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.518314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.518314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.518587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.519280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.521082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.555461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 912/1833] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:44.556560 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.585087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72288f-13e6-4906-86f7-f2369054f1f6 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.609013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.613015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2516350efd483faa420d25463ce320'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:44.614450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.615754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2516350efd483faa420d25463ce320'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:44.616129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.616767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] PATH_INFO: `/v3/projects/8ea9043a04b44cebabd4de29bbe3328e/users/b256672e4e6847ff87eb6ae91ba63a6b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.656043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea9043a04b44cebabd4de29bbe3328e, user_id=b256672e4e6847ff87eb6ae91ba63a6b, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.659294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.708874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2516350efd483faa420d25463ce320'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:44.740493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2516350efd483faa420d25463ce320'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:44.741322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2516350efd483faa420d25463ce320'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:44.752644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 913/1834] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:44 2022] PUT /identity/v3/projects/8ea9043a04b44cebabd4de29bbe3328e/users/b256672e4e6847ff87eb6ae91ba63a6b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:44.752644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.752644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:44.772787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 922/1835] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:44 2022] DELETE /identity/v3/users/2b2516350efd483faa420d25463ce320 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:44.775076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:44.777225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:44.871856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f80bb-5f0a-4292-a63a-a1831fbbfc14 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.951539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3025aa5-7799-4837-84de-4819ee38d379 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:44.959126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c9671-56d1-4f0d-bbc1-31e692de8dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.960029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c9671-56d1-4f0d-bbc1-31e692de8dde admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.961196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c9671-56d1-4f0d-bbc1-31e692de8dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.961671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c9671-56d1-4f0d-bbc1-31e692de8dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.964259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c3a76-b780-475a-9eac-913a68bb3ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:44.964434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c9671-56d1-4f0d-bbc1-31e692de8dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.964523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3a76-b780-475a-9eac-913a68bb3ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.964523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3a76-b780-475a-9eac-913a68bb3ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.964523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3a76-b780-475a-9eac-913a68bb3ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:44.964523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c3a76-b780-475a-9eac-913a68bb3ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:44.964925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c9671-56d1-4f0d-bbc1-31e692de8dde admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.972551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c3a76-b780-475a-9eac-913a68bb3ce8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:44.991106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 914/1836] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:44.995631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ececf36-ee02-490a-80a6-610890cce6f1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:44.996314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ececf36-ee02-490a-80a6-610890cce6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:44.998433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ececf36-ee02-490a-80a6-610890cce6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.010042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 923/1837] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:45.012493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296cdef-2f63-4a29-be9b-2232b0c89229 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:45.012493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296cdef-2f63-4a29-be9b-2232b0c89229 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:45.015922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296cdef-2f63-4a29-be9b-2232b0c89229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.058003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2ececf36-ee02-490a-80a6-610890cce6f1 None None] MFA Rules not processed for user `b256672e4e6847ff87eb6ae91ba63a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:45.059287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2ececf36-ee02-490a-80a6-610890cce6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:45.068617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-0296cdef-2f63-4a29-be9b-2232b0c89229 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:45.069924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0296cdef-2f63-4a29-be9b-2232b0c89229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:45.086852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 924/1838] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:45.099206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296cdef-2f63-4a29-be9b-2232b0c89229 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:45.115957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296cdef-2f63-4a29-be9b-2232b0c89229 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:45.122745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:45.123488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:45.123834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:45.124268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] PATH_INFO: `/v3/projects/4212f3c25d364a3dbf558d5c8e61c000/users/e34bab1976e5436abce0cb18a06580cc/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.145700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4212f3c25d364a3dbf558d5c8e61c000, user_id=e34bab1976e5436abce0cb18a06580cc, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:45.148536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:45.256365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 915/1839] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:45.257450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ececf36-ee02-490a-80a6-610890cce6f1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:45.274485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 925/1840] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:45 2022] PUT /identity/v3/projects/4212f3c25d364a3dbf558d5c8e61c000/users/e34bab1976e5436abce0cb18a06580cc/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:45.278350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:45.281815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:45.292542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:45.293297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ececf36-ee02-490a-80a6-610890cce6f1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:45.354232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:45.481350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:45.482056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:45.482338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:45.482682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] PATH_INFO: `/v3/projects/23e1a4563a684d5bb448a161474d8a1f/users/6a1fb6cf9a8a46bbb18657ec94d3a759/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.498065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1339f2-ac82-4f48-8f09-606c9fb6a8f1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:45.510679 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e1a4563a684d5bb448a161474d8a1f, user_id=6a1fb6cf9a8a46bbb18657ec94d3a759, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:45.513374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:45.542799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:45.543478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:45.543744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:45.544023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.580151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:45.584750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:45.599844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 916/1841] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:45 2022] PUT /identity/v3/projects/23e1a4563a684d5bb448a161474d8a1f/users/6a1fb6cf9a8a46bbb18657ec94d3a759/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:45.599844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:45.608084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:45.632757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 926/1842] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:45.634901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:45.667480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b9e5b-a119-48d1-a422-a3697d5bff61 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:45.789473 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ee4e5-c7a9-4500-aea7-a9ea28719a9b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:45.829514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc611fb4-73ef-4435-bda9-8256d84e8691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:45.830784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc611fb4-73ef-4435-bda9-8256d84e8691 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:45.831302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc611fb4-73ef-4435-bda9-8256d84e8691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:45.831820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc611fb4-73ef-4435-bda9-8256d84e8691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.835423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc611fb4-73ef-4435-bda9-8256d84e8691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:45.837440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc611fb4-73ef-4435-bda9-8256d84e8691 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:45.839559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:45.843356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:45.844743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:45.849433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.849433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:45.849433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:45.863787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 927/1843] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:45.865234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee5cfe-31ff-44b9-9736-3c83a8c1053e None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:45.865530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee5cfe-31ff-44b9-9736-3c83a8c1053e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:45.865805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee5cfe-31ff-44b9-9736-3c83a8c1053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:45.946441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-31ee5cfe-31ff-44b9-9736-3c83a8c1053e None None] MFA Rules not processed for user `6a1fb6cf9a8a46bbb18657ec94d3a759`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:45.947717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-31ee5cfe-31ff-44b9-9736-3c83a8c1053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:45.972799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 917/1844] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:45 2022] POST /identity/v3/projects => generated 385 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:45.973649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:46.121296 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 928/1845] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:46.124842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee5cfe-31ff-44b9-9736-3c83a8c1053e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:46.160986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237585b-b6d0-4a75-829d-4448b2bb5b8f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:46.176299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:46.177703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.178003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.178286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] PATH_INFO: `/v3/projects/4212f3c25d364a3dbf558d5c8e61c000/users/e34bab1976e5436abce0cb18a06580cc/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.182642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee5cfe-31ff-44b9-9736-3c83a8c1053e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:46.188433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4212f3c25d364a3dbf558d5c8e61c000, user_id=e34bab1976e5436abce0cb18a06580cc, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:46.193672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:46.201698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4cd46-a535-40df-9ea1-3cdda7408673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:46.202391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4cd46-a535-40df-9ea1-3cdda7408673 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.202642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4cd46-a535-40df-9ea1-3cdda7408673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.202922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4cd46-a535-40df-9ea1-3cdda7408673 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.203717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4cd46-a535-40df-9ea1-3cdda7408673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:46.210709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4cd46-a535-40df-9ea1-3cdda7408673 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:46.219837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 918/1846] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:45 2022] PUT /identity/v3/projects/4212f3c25d364a3dbf558d5c8e61c000/users/e34bab1976e5436abce0cb18a06580cc/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:46.222829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:46.372235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 929/1847] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:46 2022] POST /identity/v3/users => generated 299 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:46.374574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c745e9-b6cd-4196-8f81-97feb69e3775 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.375727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c745e9-b6cd-4196-8f81-97feb69e3775 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.376015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c745e9-b6cd-4196-8f81-97feb69e3775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.380403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b815b4-2ced-481a-a1ca-dab4db4d351a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:46.396212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ce918-21a3-4a5f-b0d9-29e9f9599a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:46.396212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ce918-21a3-4a5f-b0d9-29e9f9599a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.398228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ce918-21a3-4a5f-b0d9-29e9f9599a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.398539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ce918-21a3-4a5f-b0d9-29e9f9599a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.399775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ce918-21a3-4a5f-b0d9-29e9f9599a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:46.402901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ce918-21a3-4a5f-b0d9-29e9f9599a32 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:46.443294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:46.515683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b7c745e9-b6cd-4196-8f81-97feb69e3775 None None] MFA Rules not processed for user `e34bab1976e5436abce0cb18a06580cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:46.517216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b7c745e9-b6cd-4196-8f81-97feb69e3775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:46.568505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 919/1848] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:46 2022] POST /identity/v3/projects => generated 369 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:46.570993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eb43d-f855-47d9-81bc-80713d815cd4 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.573762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eb43d-f855-47d9-81bc-80713d815cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.574731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802eb43d-f855-47d9-81bc-80713d815cd4 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.578832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 920/1849] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:13:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:46.580916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eb43d-f855-47d9-81bc-80713d815cd4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:46.700053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 930/1850] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:46.705837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c745e9-b6cd-4196-8f81-97feb69e3775 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:46.741150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c745e9-b6cd-4196-8f81-97feb69e3775 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:46.822853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802eb43d-f855-47d9-81bc-80713d815cd4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:46.832777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80381675-41ed-46b4-924e-25143402926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:46.833562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381675-41ed-46b4-924e-25143402926e admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.835006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381675-41ed-46b4-924e-25143402926e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.835398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381675-41ed-46b4-924e-25143402926e admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.836305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80381675-41ed-46b4-924e-25143402926e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:46.838105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80381675-41ed-46b4-924e-25143402926e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:46.869973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:46.871060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.871520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.871941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.873120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:46.878655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 921/1851] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:46.878655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:46.878655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:46.878655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:46.880830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:46.902865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:46.905745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:46.994793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 922/1852] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:13:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:46.995890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:46.998060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:47.087387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 931/1853] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:46 2022] POST /identity/v3/users => generated 291 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:47.088299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.097229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07399735-fb94-4965-8bd0-6844a05d3ccf None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.103771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.110593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.110593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.110593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.119992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc826a-7c35-44bf-8dc9-315eee15b078 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.133851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.134563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.146671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.146671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] PATH_INFO: `/v3/projects/5a9f0d5b491d4de08e774b506321345b/users/e580431b73bb42f48b8defda0e33e63e/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.190422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9f0d5b491d4de08e774b506321345b, user_id=e580431b73bb42f48b8defda0e33e63e, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:47.194365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:47.194664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:47.196352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:47.213267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 923/1854] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:47.214052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.227876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:47.262800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c236775-ab02-4d38-9fb4-3044c5a7f8fc service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.270090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.271315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.271841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.272117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.281017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 932/1855] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:47 2022] PUT /identity/v3/projects/5a9f0d5b491d4de08e774b506321345b/users/e580431b73bb42f48b8defda0e33e63e/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:47.281999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.290662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:47.292637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:47.309448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 924/1856] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:47.310521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.334241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d013cc-5892-4fc0-ba8c-fa2cc86f90fe service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.512525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.514083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.514508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.514927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ea25e-fa4b-4f76-ac7b-4f2982724425 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.515265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.517528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:47.519393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:47.524481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.526733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:47.527602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.528572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.528572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.529457 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:47.531776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:47.550854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 925/1857] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:47.553303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.555025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 933/1858] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:47.556350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.557318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:47.559227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:47.578970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8ccaf-ae7f-4215-bfab-2d178e431d3f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.588019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.599477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.599477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.599477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] PATH_INFO: `/v3/projects/4e930d1e53c54b659a825aa1784bae67` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.599477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e930d1e53c54b659a825aa1784bae67)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:47.599477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:47.634284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fa320-c6c8-4c7c-870d-40663ad6cd4a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.640723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.641633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.642112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.642598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.645013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:47.729029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:47.731881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:47.769878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e930d1e53c54b659a825aa1784bae67', 'request_id': 'req-2c044893-d761-4a16-b14b-601646fa9b69'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:47.772827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 934/1859] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:47.774694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.781539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e930d1e53c54b659a825aa1784bae67', 'request_id': 'req-2c044893-d761-4a16-b14b-601646fa9b69'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:47.792037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e930d1e53c54b659a825aa1784bae67', 'request_id': 'req-2c044893-d761-4a16-b14b-601646fa9b69'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:47.803019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c916a-4724-4f70-9293-a925c381a24a service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:47.886079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] The token cache is being invalidate because project 4e930d1e53c54b659a825aa1784bae67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:47.886295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:47.889304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:47.891343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 926/1860] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:47 2022] DELETE /identity/v3/projects/4e930d1e53c54b659a825aa1784bae67 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:47.892180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:47.895238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:47.953181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:47.953939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:47.954226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:47.954507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] PATH_INFO: `/v3/projects/c42d9d3ad9c64a768246fed6b93281a4/users/8664555135cc4a8baabcdbafb9b3d3cb/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:47.991732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42d9d3ad9c64a768246fed6b93281a4, user_id=8664555135cc4a8baabcdbafb9b3d3cb, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:48.000010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:48.040095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c044893-d761-4a16-b14b-601646fa9b69 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:48.059472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:48.062853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:48.064169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:48.064451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] PATH_INFO: `/v3/projects/5a9f0d5b491d4de08e774b506321345b/users/e580431b73bb42f48b8defda0e33e63e/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:48.071650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9f0d5b491d4de08e774b506321345b, user_id=e580431b73bb42f48b8defda0e33e63e, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:48.079422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:48.134989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 935/1861] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:47 2022] PUT /identity/v3/projects/c42d9d3ad9c64a768246fed6b93281a4/users/8664555135cc4a8baabcdbafb9b3d3cb/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:48.135115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 927/1862] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:47 2022] PUT /identity/v3/projects/5a9f0d5b491d4de08e774b506321345b/users/e580431b73bb42f48b8defda0e33e63e/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:48.135203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:48.135203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:48.135342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:48.135342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:48.262411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9ac95-0574-4188-8aa9-4ec99afb2898 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:48.271431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:48.272858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:48.273546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:48.274230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] PATH_INFO: `/v3/users/90e0893b50904214bbc046fe3fe9722b` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:48.275852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e0893b50904214bbc046fe3fe9722b)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:48.283646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:48.462656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2695687-a8f5-48a1-8f68-b8ce7f7b6d03 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:48.486172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:48.490692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136847e-2c13-4442-b769-323aaf6c5a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:48.498523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136847e-2c13-4442-b769-323aaf6c5a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:48.498662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136847e-2c13-4442-b769-323aaf6c5a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:48.498955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136847e-2c13-4442-b769-323aaf6c5a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:48.505244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136847e-2c13-4442-b769-323aaf6c5a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:48.530261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136847e-2c13-4442-b769-323aaf6c5a37 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:48.551513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e0893b50904214bbc046fe3fe9722b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:48.551513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e0893b50904214bbc046fe3fe9722b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:48.601665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 928/1863] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:13:48.606662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:48.608963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:48.608963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:48.664357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e0893b50904214bbc046fe3fe9722b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:48.702884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e0893b50904214bbc046fe3fe9722b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:48.705913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e0893b50904214bbc046fe3fe9722b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:48.741532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 936/1864] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:48 2022] DELETE /identity/v3/users/90e0893b50904214bbc046fe3fe9722b => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:48.743378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:48.745511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:48.749513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] MFA Rules not processed for user `e580431b73bb42f48b8defda0e33e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:48.765320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:48.879497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:48.915659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30517e46-63f2-4724-ba17-c73681388acf admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:48.927751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee4e36-e9f0-411f-8fb6-044b6946b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:48.929852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee4e36-e9f0-411f-8fb6-044b6946b73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:48.930193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee4e36-e9f0-411f-8fb6-044b6946b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:48.930528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee4e36-e9f0-411f-8fb6-044b6946b73a admin admin] PATH_INFO: `/v3/projects/c42d9d3ad9c64a768246fed6b93281a4/users/8664555135cc4a8baabcdbafb9b3d3cb/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:48.936365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee4e36-e9f0-411f-8fb6-044b6946b73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42d9d3ad9c64a768246fed6b93281a4, user_id=8664555135cc4a8baabcdbafb9b3d3cb, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:48.943812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee4e36-e9f0-411f-8fb6-044b6946b73a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:48.981193 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 929/1866] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:48.981397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 937/1866] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:48 2022] PUT /identity/v3/projects/c42d9d3ad9c64a768246fed6b93281a4/users/8664555135cc4a8baabcdbafb9b3d3cb/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:48.989146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:48.989443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:48.989723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:48.998980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:49.000523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:49.100251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] MFA Rules not processed for user `8664555135cc4a8baabcdbafb9b3d3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:49.101606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:49.163625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69774c-66bd-4185-bd44-9b4ee7ce75d7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:49.205665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 938/1867] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:49.223477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:49.225839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:49.284850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:49.285496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:49.285782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:49.286095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:49.408469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:49.410207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:49.434021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 930/1868] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:49.489027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7854fa8-7431-42b9-a6d5-d4445e44e662 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:49.497484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:49.498278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:49.499632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:49.499977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:49.546712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:49.550935 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:49.551722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:49.555313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:49.591553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 939/1869] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:49 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:49.665483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80bfac-fc2e-48e6-8e65-a560f9788eba service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:49.673423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:49.674500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:49.675137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:49.675886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] PATH_INFO: `/v3/users/b256672e4e6847ff87eb6ae91ba63a6b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:49.685127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b256672e4e6847ff87eb6ae91ba63a6b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:49.694699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:49.708940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:49.727613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f82a59-a49b-464e-913c-3071c10afca3 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:49.736728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550c346-f9d8-4a03-9bbd-83ee5d7e507a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:49.738608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c346-f9d8-4a03-9bbd-83ee5d7e507a service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:49.740920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c346-f9d8-4a03-9bbd-83ee5d7e507a service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:49.742228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c346-f9d8-4a03-9bbd-83ee5d7e507a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:49.754411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2550c346-f9d8-4a03-9bbd-83ee5d7e507a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:49.950695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550c346-f9d8-4a03-9bbd-83ee5d7e507a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:49.952154 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550c346-f9d8-4a03-9bbd-83ee5d7e507a service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:49.957592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b256672e4e6847ff87eb6ae91ba63a6b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:49.959231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b256672e4e6847ff87eb6ae91ba63a6b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:49.979276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 940/1870] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:49.981343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590ea2-997f-4c93-996c-5313fdd13887 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:49.982247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590ea2-997f-4c93-996c-5313fdd13887 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:49.982759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7590ea2-997f-4c93-996c-5313fdd13887 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:49.985854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 941/1871] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:13:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:13:50.043998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b256672e4e6847ff87eb6ae91ba63a6b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.065862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b256672e4e6847ff87eb6ae91ba63a6b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.090118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d9d551da-bc95-4708-9e5e-aa57eb706f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b256672e4e6847ff87eb6ae91ba63a6b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.107932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 931/1872] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:49 2022] DELETE /identity/v3/users/b256672e4e6847ff87eb6ae91ba63a6b => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:50.113099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:50.113911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:50.113911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:50.154495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:50.155757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:50.181049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:50.181367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:50.181617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:50.275338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:50.277147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:50.356338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 942/1873] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:50.365850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:50.371370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:50.395149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b57ec-6b2b-457e-b777-372b9ad19ee7 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:50.402102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:50.403421 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:50.403992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:50.404448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] PATH_INFO: `/v3/projects/3726e69ea04046eebe548b4167385ee6` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:50.405888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3726e69ea04046eebe548b4167385ee6)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:50.408036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:50.496987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 932/1874] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:50.498274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:50.499813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:50.571631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3726e69ea04046eebe548b4167385ee6', 'request_id': 'req-4c978f64-844c-4ba3-9b94-e5fc55ac063d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.599040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd207b7-d147-4ec3-8a93-aa4372998890 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:50.608765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3726e69ea04046eebe548b4167385ee6', 'request_id': 'req-4c978f64-844c-4ba3-9b94-e5fc55ac063d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.649687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3726e69ea04046eebe548b4167385ee6', 'request_id': 'req-4c978f64-844c-4ba3-9b94-e5fc55ac063d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.677750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:50.678541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:50.678769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:50.679106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:50.732646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:50.744494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] The token cache is being invalidate because project 3726e69ea04046eebe548b4167385ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:50.753134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.754333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-4c978f64-844c-4ba3-9b94-e5fc55ac063d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:50.758256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 943/1875] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:50 2022] DELETE /identity/v3/projects/3726e69ea04046eebe548b4167385ee6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:50.763655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df33f1-398b-4eab-8fda-c4d652e0fe93 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:50.764020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df33f1-398b-4eab-8fda-c4d652e0fe93 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:50.764435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df33f1-398b-4eab-8fda-c4d652e0fe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:50.808702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:50.810517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-00df33f1-398b-4eab-8fda-c4d652e0fe93 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:50.811730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:50.818169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-00df33f1-398b-4eab-8fda-c4d652e0fe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:50.835333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 944/1876] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:50.835865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 933/1877] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:50.844523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa202fc3-472c-4fe9-a763-93e52eff0027 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:50.845203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa202fc3-472c-4fe9-a763-93e52eff0027 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:50.846177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa202fc3-472c-4fe9-a763-93e52eff0027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:50.871862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:50.906129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-fa202fc3-472c-4fe9-a763-93e52eff0027 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:50.911214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fa202fc3-472c-4fe9-a763-93e52eff0027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:50.932776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 945/1878] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:50.937209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:50.939361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:50.963716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887a20-7f3b-4027-8411-7d52e9822b24 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:50.969052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:50.970076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:50.970467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:50.970802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:50.971708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:50.972542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:50.975310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.014181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 934/1879] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:13:51.017298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa202fc3-472c-4fe9-a763-93e52eff0027 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:51.032968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa202fc3-472c-4fe9-a763-93e52eff0027 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:51.066261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:51.066902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.067190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.067486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:51.068226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:51.073372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.147423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:51.151078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:51.220465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 946/1880] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:51 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:51.222896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:51.321527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925554b-16f1-484d-9567-e232bba8160b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:51.412992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece062-f9c9-430d-b67d-c294925121dd admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:51.426011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:51.426711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.427001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.427280 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:51.427969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:51.429950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.435837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:51.442558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.442558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.442558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:51.442558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:51.487518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:51.488135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:51.495577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.517197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 935/1881] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:51.525168 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 947/1882] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:51 2022] POST /identity/v3/users => generated 288 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:51.526299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:51.562782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19991914-94b5-46b0-9123-cce6efa12467 service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:51.581323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:51.582201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.583268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.583706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:51.585418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:51.590855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.615403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 936/1883] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:51.618572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:51.637243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394c332-4b2f-4664-9ead-5adc3cb4238d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:51.651515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:51.654856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.654856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.654856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:51.654856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:51.656676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.679924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 948/1884] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:51.684512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:51.724443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d84ff9-52a5-4e20-9b25-669a9ff13d4a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:51.737662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942e9c1-dccf-4848-831f-4c53a0a0933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:51.738367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e9c1-dccf-4848-831f-4c53a0a0933f admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.738649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e9c1-dccf-4848-831f-4c53a0a0933f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.741438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e9c1-dccf-4848-831f-4c53a0a0933f admin admin] PATH_INFO: `/v3/projects/785f72fbecb042cab2a97bc5193c15d3/users/03e4ddeb79274bd1a19534d467e01657/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:51.776697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942e9c1-dccf-4848-831f-4c53a0a0933f admin admin] RBAC: Authorizing `identity:create_grant(project_id=785f72fbecb042cab2a97bc5193c15d3, user_id=03e4ddeb79274bd1a19534d467e01657, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:51.783721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942e9c1-dccf-4848-831f-4c53a0a0933f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.800410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:51.808176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:51.844864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ece211-6346-44c3-b017-27784f5ca9d4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:51.855537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:51.858858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.859438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.860656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] PATH_INFO: `/v3/projects/8ea9043a04b44cebabd4de29bbe3328e` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:51.863748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea9043a04b44cebabd4de29bbe3328e)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:51.866772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:51.897336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 937/1885] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:51 2022] PUT /identity/v3/projects/785f72fbecb042cab2a97bc5193c15d3/users/03e4ddeb79274bd1a19534d467e01657/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:51.902683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af04ee-fc2f-43a4-b0cc-0fb09e07ae96 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:51.904261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af04ee-fc2f-43a4-b0cc-0fb09e07ae96 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:51.905180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af04ee-fc2f-43a4-b0cc-0fb09e07ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:52.012082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-00af04ee-fc2f-43a4-b0cc-0fb09e07ae96 None None] MFA Rules not processed for user `03e4ddeb79274bd1a19534d467e01657`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:52.017423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-00af04ee-fc2f-43a4-b0cc-0fb09e07ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:52.030506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea9043a04b44cebabd4de29bbe3328e', 'request_id': 'req-059f8ce2-3da2-4b98-9c92-dd1861575bc1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.040528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea9043a04b44cebabd4de29bbe3328e', 'request_id': 'req-059f8ce2-3da2-4b98-9c92-dd1861575bc1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.049692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea9043a04b44cebabd4de29bbe3328e', 'request_id': 'req-059f8ce2-3da2-4b98-9c92-dd1861575bc1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.162526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 938/1886] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:52.169933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af04ee-fc2f-43a4-b0cc-0fb09e07ae96 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:52.211037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] The token cache is being invalidate because project 8ea9043a04b44cebabd4de29bbe3328e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:52.212678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.216069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.222555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 949/1887] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:51 2022] DELETE /identity/v3/projects/8ea9043a04b44cebabd4de29bbe3328e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:52.224594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:52.241623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:52.316682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af04ee-fc2f-43a4-b0cc-0fb09e07ae96 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:52.338767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:52.339311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:52.339598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:52.348903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:52.349868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:52.361727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:52.458715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f8ce2-3da2-4b98-9c92-dd1861575bc1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:52.465974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:52.467082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:52.467791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:52.468208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] PATH_INFO: `/v3/users/e580431b73bb42f48b8defda0e33e63e` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:52.469820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e580431b73bb42f48b8defda0e33e63e)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:52.472274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:52.495226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 939/1888] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:52 2022] POST /identity/v3/projects => generated 377 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:52.501987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:52.504479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:52.631068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210718b3-2890-4276-9c22-6135bf7af833 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:52.641855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:52.643805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:52.644191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:52.644551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:52.646824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:52.659681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:52.797780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 940/1889] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:52 2022] POST /identity/v3/users => generated 295 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:52.801648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:52.811039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e580431b73bb42f48b8defda0e33e63e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.813628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e580431b73bb42f48b8defda0e33e63e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.815243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c3fa5-ed55-4514-93f2-1bfbc1ad2610 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:52.832094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:52.833841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:52.834149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:52.834783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:52.835501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:52.842601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:52.878565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e580431b73bb42f48b8defda0e33e63e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.886581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 941/1890] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:13:52.888331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:52.907577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e580431b73bb42f48b8defda0e33e63e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.911830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e580431b73bb42f48b8defda0e33e63e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:52.924295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e885f-981d-4e99-a21e-0d1e85231148 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:52.936029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 950/1891] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:52 2022] DELETE /identity/v3/users/e580431b73bb42f48b8defda0e33e63e => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:52.944893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:52.947066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:52.947526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:52.948211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] PATH_INFO: `/v3/projects/4ca56bfd0e364aa8857e677f5d83fbb6/users/6b4bcdb325794170869bbfe6ad5c97af/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:52.969306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca56bfd0e364aa8857e677f5d83fbb6, user_id=6b4bcdb325794170869bbfe6ad5c97af, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:52.974100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:53.491003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 942/1892] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:52 2022] PUT /identity/v3/projects/4ca56bfd0e364aa8857e677f5d83fbb6/users/6b4bcdb325794170869bbfe6ad5c97af/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:53.627177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:53.636887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:53.724258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3484a4-18c8-4400-aef4-084ecf24b4b9 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:53.787072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ab90b-1059-4cf6-ac09-abdd8932a8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:53.787708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ab90b-1059-4cf6-ac09-abdd8932a8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:53.791010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ab90b-1059-4cf6-ac09-abdd8932a8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:53.791508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ab90b-1059-4cf6-ac09-abdd8932a8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:53.792249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ab90b-1059-4cf6-ac09-abdd8932a8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:53.793904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ab90b-1059-4cf6-ac09-abdd8932a8cb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:53.807255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 951/1893] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:53.811126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:53.824077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262b12f-b497-4e48-8bb2-26d941408bd9 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:53.840858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:53.840858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:53.840858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:53.840858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] PATH_INFO: `/v3/projects/4ca56bfd0e364aa8857e677f5d83fbb6/users/6b4bcdb325794170869bbfe6ad5c97af/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:53.841348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca56bfd0e364aa8857e677f5d83fbb6, user_id=6b4bcdb325794170869bbfe6ad5c97af, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:53.842565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:53.857979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 943/1894] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:53 2022] PUT /identity/v3/projects/4ca56bfd0e364aa8857e677f5d83fbb6/users/6b4bcdb325794170869bbfe6ad5c97af/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:53.869215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:53.870073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:53.870484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:53.906150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] MFA Rules not processed for user `6b4bcdb325794170869bbfe6ad5c97af`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:53.907982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:53.994200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 952/1895] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:54.458979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:54.462381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:54.583250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:54.585653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8f6f-fea2-45cc-b08f-6ea7adec715f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:54.590706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:54.591355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:54.591640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:54.591948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:54.594196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:54.698338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:54.699782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:54.712508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 944/1896] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:54.768557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:54.775991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:54.846144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:54.853409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:54.855042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:54.904666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbf234-a4d8-4b39-8f68-c0bf4c2d39aa None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:54.910254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:54.911083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:54.911522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:54.911938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] PATH_INFO: `/v3/projects/5a9f0d5b491d4de08e774b506321345b` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:54.913620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a9f0d5b491d4de08e774b506321345b)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:54.917751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:54.983953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed97ef-1bd1-4e96-a95d-4804954930ab service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:54.989578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:54.990235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:54.990768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:54.990963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] PATH_INFO: `/v3/users/d28d55cc6b1548a9b87fac2f0d509f13` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:54.992062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28d55cc6b1548a9b87fac2f0d509f13)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:54.995782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.085806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9f0d5b491d4de08e774b506321345b', 'request_id': 'req-df8fd1b6-37a2-43ca-874a-9d958f3c7397'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.091999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9f0d5b491d4de08e774b506321345b', 'request_id': 'req-df8fd1b6-37a2-43ca-874a-9d958f3c7397'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.097445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9f0d5b491d4de08e774b506321345b', 'request_id': 'req-df8fd1b6-37a2-43ca-874a-9d958f3c7397'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.191351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] The token cache is being invalidate because project 5a9f0d5b491d4de08e774b506321345b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:55.195563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.195563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-df8fd1b6-37a2-43ca-874a-9d958f3c7397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.195563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 953/1897] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:54 2022] DELETE /identity/v3/projects/5a9f0d5b491d4de08e774b506321345b => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:55.198713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63d8f-51db-4ece-ae60-5d76f7e56ecf None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.199170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63d8f-51db-4ece-ae60-5d76f7e56ecf None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.199446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63d8f-51db-4ece-ae60-5d76f7e56ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.228437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a0b63d8f-51db-4ece-ae60-5d76f7e56ecf None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:55.230525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a0b63d8f-51db-4ece-ae60-5d76f7e56ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:55.235082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28d55cc6b1548a9b87fac2f0d509f13'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.235813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28d55cc6b1548a9b87fac2f0d509f13'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.297515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 954/1898] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:55.305324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b355f-afdf-4a0f-a056-7edc42304555 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.305610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b355f-afdf-4a0f-a056-7edc42304555 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.305888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b355f-afdf-4a0f-a056-7edc42304555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.325026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-384b355f-afdf-4a0f-a056-7edc42304555 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:55.329152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-384b355f-afdf-4a0f-a056-7edc42304555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:55.334445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 955/1899] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:55.344613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b355f-afdf-4a0f-a056-7edc42304555 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:55.354133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28d55cc6b1548a9b87fac2f0d509f13'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.354467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b355f-afdf-4a0f-a056-7edc42304555 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:55.359713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:55.360343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.363305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.363305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.363305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:55.363305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:55.363787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.367766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28d55cc6b1548a9b87fac2f0d509f13'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.374881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd8581c9-2f5a-4c89-8fba-8ffe1fe4dcd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28d55cc6b1548a9b87fac2f0d509f13'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:55.379849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 956/1900] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:13:55.381660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 945/1901] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:54 2022] DELETE /identity/v3/users/d28d55cc6b1548a9b87fac2f0d509f13 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:55.386486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:55.386486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:55.390006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.391228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.391755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.397362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140745e-30ff-4dc3-b116-8e8bb8a61c7a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:55.412335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:55.414883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:55.454074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:55.454694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.454967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.455238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.455898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:55.457830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.474169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 946/1902] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:55.483877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:55.485004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:55.552859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db707f1-ffae-43ef-81d2-30475fe48142 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:55.557224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:55.557851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.558136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.558412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.567770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:55.575672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.578550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 957/1903] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:55 2022] POST /identity/v3/projects => generated 391 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:13:55.581766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 947/1904] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:55.582946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:55.656576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c16a21-867f-4a00-a135-1529784bdfd0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:55.662146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:55.662773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.663051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.663322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.664739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:55.668452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.751845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 958/1905] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:55 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:55.755957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:55.766753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806ed3-6b83-443d-bed9-3a315da4f1f9 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:55.771959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:55.775241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.775241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.775241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.775241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:55.776343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.791536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 948/1906] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:55.796493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:55.810902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7afdce-e98f-40bb-9fb7-e37627bd19e9 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:55.817513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0be89-1dfb-48b9-ad7a-023bcbf66c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:55.818489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0be89-1dfb-48b9-ad7a-023bcbf66c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.818932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0be89-1dfb-48b9-ad7a-023bcbf66c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.819406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0be89-1dfb-48b9-ad7a-023bcbf66c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.820363 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0be89-1dfb-48b9-ad7a-023bcbf66c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:55.822428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0be89-1dfb-48b9-ad7a-023bcbf66c08 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.840818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 959/1907] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:55.842850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:55.854666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1e43f-733b-4c32-8e36-05c8cd39dbac admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:55.861005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:55.861697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.861982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.862438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] PATH_INFO: `/v3/projects/6c503ba498444f59b55f7127316a5fbf/users/8c32d86f55a54b00a912caa740a20f52/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:55.882474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c503ba498444f59b55f7127316a5fbf, user_id=8c32d86f55a54b00a912caa740a20f52, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:55.886756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:55.974828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 949/1908] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:55 2022] PUT /identity/v3/projects/6c503ba498444f59b55f7127316a5fbf/users/8c32d86f55a54b00a912caa740a20f52/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:55.985411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ada9f-d984-4bae-9d37-cf64e69dda79 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:55.986000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ada9f-d984-4bae-9d37-cf64e69dda79 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:55.986000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ada9f-d984-4bae-9d37-cf64e69dda79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.016871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5d8ada9f-d984-4bae-9d37-cf64e69dda79 None None] MFA Rules not processed for user `8c32d86f55a54b00a912caa740a20f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:56.018320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5d8ada9f-d984-4bae-9d37-cf64e69dda79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:56.045959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.047734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:56.115878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 960/1909] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:56.121824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ada9f-d984-4bae-9d37-cf64e69dda79 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.123438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6960b3-a15d-4de7-9b64-ba1bc6eb46ec admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.128195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.129024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.132372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.132372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] PATH_INFO: `/v3/projects/62a5eed2e9c441f9835b1cc4ab0bb74a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.132372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a5eed2e9c441f9835b1cc4ab0bb74a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.132662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ada9f-d984-4bae-9d37-cf64e69dda79 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.132743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.136943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.137555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.137844 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.138121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.138789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.140752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.251272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 961/1910] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:56 2022] POST /identity/v3/projects => generated 391 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:56.255330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.261770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a5eed2e9c441f9835b1cc4ab0bb74a', 'request_id': 'req-f18deb5a-3070-4318-8e74-99ee2172713c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.267904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a5eed2e9c441f9835b1cc4ab0bb74a', 'request_id': 'req-f18deb5a-3070-4318-8e74-99ee2172713c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.274793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a5eed2e9c441f9835b1cc4ab0bb74a', 'request_id': 'req-f18deb5a-3070-4318-8e74-99ee2172713c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.315764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7891652-ce9d-4a25-9fb7-09ebbdab6b70 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.320352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.321574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.321872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.322142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.322796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.324720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.369568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] The token cache is being invalidate because project 62a5eed2e9c441f9835b1cc4ab0bb74a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:56.373683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.373683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.373683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 950/1911] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:56 2022] DELETE /identity/v3/projects/62a5eed2e9c441f9835b1cc4ab0bb74a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:56.376048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.377424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:56.428026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 962/1912] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:56 2022] POST /identity/v3/users => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:56.432087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.433932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:56.446860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6aba5e-aae5-4f8e-aade-fdfb788c8a17 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.451770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18deb5a-3070-4318-8e74-99ee2172713c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.459351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.459533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.460437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.460537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.460692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.460992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] PATH_INFO: `/v3/users/8819a7cea22b48b88177c2e6e8fd0546` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.461929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.462425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.463384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.464295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8819a7cea22b48b88177c2e6e8fd0546)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.465193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.467952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.477399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 963/1913] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:13:56.482237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.495250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b472ac-60cb-4c66-a872-7fa91952b2df admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.500124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.500887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.501408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.501630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] PATH_INFO: `/v3/projects/2e7a617f69ce4801bcef8b79a90d3342/users/2aad885ed9c945fe8b21e1db3db51891/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.519203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7a617f69ce4801bcef8b79a90d3342, user_id=2aad885ed9c945fe8b21e1db3db51891, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.521962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.624321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 964/1914] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:56 2022] PUT /identity/v3/projects/2e7a617f69ce4801bcef8b79a90d3342/users/2aad885ed9c945fe8b21e1db3db51891/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:56.628342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.689790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58250d-8647-42b4-9cbf-6db89f657ad6 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.694437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.695095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.695401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.696204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.696938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.698526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.708791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 965/1915] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:13:56.712837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:56.722181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20938e-4360-43d1-8e5e-576fc6d9e869 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:56.731718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617bc82-9802-4cd8-b14f-be278a695d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:56.732429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617bc82-9802-4cd8-b14f-be278a695d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.734776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617bc82-9802-4cd8-b14f-be278a695d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.734776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617bc82-9802-4cd8-b14f-be278a695d3a admin admin] PATH_INFO: `/v3/projects/2e7a617f69ce4801bcef8b79a90d3342/users/2aad885ed9c945fe8b21e1db3db51891/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.735093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617bc82-9802-4cd8-b14f-be278a695d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7a617f69ce4801bcef8b79a90d3342, user_id=2aad885ed9c945fe8b21e1db3db51891, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:56.737770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617bc82-9802-4cd8-b14f-be278a695d3a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:56.740197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8819a7cea22b48b88177c2e6e8fd0546'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.740903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8819a7cea22b48b88177c2e6e8fd0546'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.749467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 966/1916] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:56 2022] PUT /identity/v3/projects/2e7a617f69ce4801bcef8b79a90d3342/users/2aad885ed9c945fe8b21e1db3db51891/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:56.763997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:56.764315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:56.764629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:56.792806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] MFA Rules not processed for user `2aad885ed9c945fe8b21e1db3db51891`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:56.794021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:56.839516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8819a7cea22b48b88177c2e6e8fd0546'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.841242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:56.853776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8819a7cea22b48b88177c2e6e8fd0546'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.860962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8819a7cea22b48b88177c2e6e8fd0546'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:56.868126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 951/1917] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:56 2022] DELETE /identity/v3/users/8819a7cea22b48b88177c2e6e8fd0546 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:56.878536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 967/1918] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:57.199713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:57.260018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:57.261577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:57.326429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc949f99-a04c-48db-83c9-d2bfef160cbc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:57.334652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:57.336624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:57.337133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:57.340726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:57.362449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:57.364120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:57.419525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:57.421036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:57.431043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 952/1919] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:57.437418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b9bb-a0ca-4979-a756-f5ce5fd40d0b None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:57.503280 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:57.503944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:57.504229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:57.504536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] PATH_INFO: `/v3/projects/4c9e9ef506404ea781887ad7507cb711` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:57.505745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9e9ef506404ea781887ad7507cb711)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:57.507692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:57.524764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:57.536750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34766700-372b-4486-8272-e3aa11ddb039 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:57.543673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:57.543673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:57.543673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:57.543673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:57.546096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:57.613002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:57.614446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:57.624545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 953/1920] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:57.648373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9e9ef506404ea781887ad7507cb711', 'request_id': 'req-b4fed6b6-9d93-4248-bfe4-e150250bb527'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:57.655475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9e9ef506404ea781887ad7507cb711', 'request_id': 'req-b4fed6b6-9d93-4248-bfe4-e150250bb527'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:57.662637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9e9ef506404ea781887ad7507cb711', 'request_id': 'req-b4fed6b6-9d93-4248-bfe4-e150250bb527'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:57.765454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] The token cache is being invalidate because project 4c9e9ef506404ea781887ad7507cb711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:57.770071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:57.770071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:57.770071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 968/1921] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:57 2022] DELETE /identity/v3/projects/4c9e9ef506404ea781887ad7507cb711 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:57.772700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:57.783918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f5d16-20c1-43de-96bf-a37ad146b724 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:57.838537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:57.839175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:57.839454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:57.839723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] PATH_INFO: `/v3/users/e34bab1976e5436abce0cb18a06580cc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:57.840929 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34bab1976e5436abce0cb18a06580cc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:57.842839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:57.950761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:57.952161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:57.965615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fed6b6-9d93-4248-bfe4-e150250bb527 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:57.970227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:57.970911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:57.971159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:57.971432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] PATH_INFO: `/v3/users/6a1fb6cf9a8a46bbb18657ec94d3a759` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:57.972635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1fb6cf9a8a46bbb18657ec94d3a759)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:57.974579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:58.080254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34bab1976e5436abce0cb18a06580cc'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.080942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34bab1976e5436abce0cb18a06580cc'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.146453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34bab1976e5436abce0cb18a06580cc'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.159300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34bab1976e5436abce0cb18a06580cc'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.159890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1fb6cf9a8a46bbb18657ec94d3a759'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.160631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1fb6cf9a8a46bbb18657ec94d3a759'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.166634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c46ae2db-47b5-47f2-ae5a-02c62fe17aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34bab1976e5436abce0cb18a06580cc'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.175826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 954/1922] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:57 2022] DELETE /identity/v3/users/e34bab1976e5436abce0cb18a06580cc => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:58.249017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1fb6cf9a8a46bbb18657ec94d3a759'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.264511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1fb6cf9a8a46bbb18657ec94d3a759'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.265079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1fb6cf9a8a46bbb18657ec94d3a759'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.282207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 969/1923] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:57 2022] DELETE /identity/v3/users/6a1fb6cf9a8a46bbb18657ec94d3a759 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:58.287639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:58.287639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:58.287639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:58.312798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:58.319551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:58.386716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 955/1924] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:58.400273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:58.400756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:58.479525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9aded-e557-4585-af3b-604662df5e16 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:58.485517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b560845-304e-42ff-a7e1-c868357943ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:58.485517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b560845-304e-42ff-a7e1-c868357943ef service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:58.485517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b560845-304e-42ff-a7e1-c868357943ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:58.485765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b560845-304e-42ff-a7e1-c868357943ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:58.498008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b560845-304e-42ff-a7e1-c868357943ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:58.499693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b560845-304e-42ff-a7e1-c868357943ef service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:58.514440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 970/1925] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:58.683868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:58.718462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:58.720239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:58.739548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc99af-b029-4a73-97b0-4dfd47b6787c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:58.759466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:58.760134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:58.760449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:58.760729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] PATH_INFO: `/v3/projects/4212f3c25d364a3dbf558d5c8e61c000` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:58.761912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4212f3c25d364a3dbf558d5c8e61c000)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:58.763847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:58.890214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:58.921972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4212f3c25d364a3dbf558d5c8e61c000', 'request_id': 'req-eb216776-0156-44d7-b6fa-b935be0bcb68'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.928402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4212f3c25d364a3dbf558d5c8e61c000', 'request_id': 'req-eb216776-0156-44d7-b6fa-b935be0bcb68'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:58.933930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4212f3c25d364a3dbf558d5c8e61c000', 'request_id': 'req-eb216776-0156-44d7-b6fa-b935be0bcb68'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.021124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] The token cache is being invalidate because project 4212f3c25d364a3dbf558d5c8e61c000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:59.026391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.026391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.026391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 956/1926] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:58 2022] DELETE /identity/v3/projects/4212f3c25d364a3dbf558d5c8e61c000 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:59.031188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449d42-7fe7-4915-8c1d-6799c6707fe6 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.031714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449d42-7fe7-4915-8c1d-6799c6707fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.031988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449d42-7fe7-4915-8c1d-6799c6707fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.052430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a1449d42-7fe7-4915-8c1d-6799c6707fe6 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:59.053650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a1449d42-7fe7-4915-8c1d-6799c6707fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:59.111916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 971/1927] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:59.121961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.122435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1064d-f5ed-44f7-8bdd-c73d70869797 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.122775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1064d-f5ed-44f7-8bdd-c73d70869797 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.122980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1064d-f5ed-44f7-8bdd-c73d70869797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.123452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:59.139010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb216776-0156-44d7-b6fa-b935be0bcb68 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.150841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bba1064d-f5ed-44f7-8bdd-c73d70869797 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:59.151864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.152101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bba1064d-f5ed-44f7-8bdd-c73d70869797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:59.152558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.152844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.153122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] PATH_INFO: `/v3/projects/23e1a4563a684d5bb448a161474d8a1f` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.154296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e1a4563a684d5bb448a161474d8a1f)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.156229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.160185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 972/1928] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:59.165002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1064d-f5ed-44f7-8bdd-c73d70869797 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.174318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1064d-f5ed-44f7-8bdd-c73d70869797 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.178951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.179555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.179875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.180145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.181039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:13:59.187155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.187155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.194608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 973/1929] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:13:59.199183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.208431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ffa55-0971-4eed-b963-209d2e505abd admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.216173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.216173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.216173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.216173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.216583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.217366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.278967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e1a4563a684d5bb448a161474d8a1f', 'request_id': 'req-d710ccd0-49f5-4b10-8385-8e4b2db3638f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.285838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e1a4563a684d5bb448a161474d8a1f', 'request_id': 'req-d710ccd0-49f5-4b10-8385-8e4b2db3638f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.291319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e1a4563a684d5bb448a161474d8a1f', 'request_id': 'req-d710ccd0-49f5-4b10-8385-8e4b2db3638f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.320915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 974/1930] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:59 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:59.324924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.375385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] The token cache is being invalidate because project 23e1a4563a684d5bb448a161474d8a1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:13:59.375385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.375385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.375385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 957/1931] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:13:59 2022] DELETE /identity/v3/projects/23e1a4563a684d5bb448a161474d8a1f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:13:59.376973 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.378374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:59.393870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314330fe-33d7-4db8-a38e-ca523003360a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.399990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.410059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.410059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.410059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.410059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.410059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.444420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710ccd0-49f5-4b10-8385-8e4b2db3638f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.449880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.451362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.451362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.451362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] PATH_INFO: `/v3/users/8664555135cc4a8baabcdbafb9b3d3cb` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.452826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8664555135cc4a8baabcdbafb9b3d3cb)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.454772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.474476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 975/1932] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:13:59 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:13:59.485783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.485783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:59.493870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df8c84-3e90-4302-b41a-ac25a7625177 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.498834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.499455 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.499740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.500127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.500809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.502404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.513384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 976/1933] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:13:59.517470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.527768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caa6f0-f883-4eeb-9fd9-6a111ded3485 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.532588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.533197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.533482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.533755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.534491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.536303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.556745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 977/1934] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:13:59.557656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.572453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a18c4c-c526-49ee-8a06-1298b37202ab admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.576054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61189882-288a-499c-a6d4-716115fda934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.576767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189882-288a-499c-a6d4-716115fda934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.577048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189882-288a-499c-a6d4-716115fda934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.577324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61189882-288a-499c-a6d4-716115fda934 admin admin] PATH_INFO: `/v3/projects/2b087605a91049d596c5f386d84f3376/users/9886343b48e84e3e931ba54a08019a3b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.595453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61189882-288a-499c-a6d4-716115fda934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b087605a91049d596c5f386d84f3376, user_id=9886343b48e84e3e931ba54a08019a3b, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.598243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61189882-288a-499c-a6d4-716115fda934 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:59.658811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8664555135cc4a8baabcdbafb9b3d3cb'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.659478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8664555135cc4a8baabcdbafb9b3d3cb'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.697354 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 978/1935] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:13:59 2022] PUT /identity/v3/projects/2b087605a91049d596c5f386d84f3376/users/9886343b48e84e3e931ba54a08019a3b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:59.708535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf067fe8-3901-49ba-b685-862060638766 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.708828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf067fe8-3901-49ba-b685-862060638766 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.709114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf067fe8-3901-49ba-b685-862060638766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.739173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bf067fe8-3901-49ba-b685-862060638766 None None] MFA Rules not processed for user `9886343b48e84e3e931ba54a08019a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:59.740431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bf067fe8-3901-49ba-b685-862060638766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:59.781965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8664555135cc4a8baabcdbafb9b3d3cb'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.794693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8664555135cc4a8baabcdbafb9b3d3cb'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.801897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8664555135cc4a8baabcdbafb9b3d3cb'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:13:59.802921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 979/1936] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:13:59.807911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf067fe8-3901-49ba-b685-862060638766 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:59.808403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 958/1937] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:13:59 2022] DELETE /identity/v3/users/8664555135cc4a8baabcdbafb9b3d3cb => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:13:59.870593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf067fe8-3901-49ba-b685-862060638766 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:59.913325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:59.931018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:59.931638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:59.931914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:59.932221 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:59.932947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:59.934829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.041604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 980/1938] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:13:59 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:00.045632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.048116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.066440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dada6-5b4c-423d-ba14-554f562c4cc5 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.163551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.164333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.164745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.165116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.165901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.167932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.169547 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:00.286648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 959/1939] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:00 2022] POST /identity/v3/users => generated 299 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:00.287828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.298191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c20c47-1623-4faa-85eb-1ce16a86e6c0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.302841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.303503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.303860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.304141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.304910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.306507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.319548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 981/1940] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:00.321224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.323191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.327212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.344057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ccabe-22ef-44a8-b5a0-d1443e688f62 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.344794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d01786-c8b4-4ad8-9234-40b99a4b7e7c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.349052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.349898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.350168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.350448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] PATH_INFO: `/v3/projects/c42d9d3ad9c64a768246fed6b93281a4` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.350864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.351801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.351897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42d9d3ad9c64a768246fed6b93281a4)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.352249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.352656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] PATH_INFO: `/v3/projects/b9c65645583241288a9808e3a531b3a4/users/25f0ee3f17e642db841c247eca04941a/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.353758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.371951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c65645583241288a9808e3a531b3a4, user_id=25f0ee3f17e642db841c247eca04941a, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.375116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.464419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 982/1941] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:00 2022] PUT /identity/v3/projects/b9c65645583241288a9808e3a531b3a4/users/25f0ee3f17e642db841c247eca04941a/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:00.464419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.465119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42d9d3ad9c64a768246fed6b93281a4', 'request_id': 'req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:00.474997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42d9d3ad9c64a768246fed6b93281a4', 'request_id': 'req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:00.481486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42d9d3ad9c64a768246fed6b93281a4', 'request_id': 'req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:00.542160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4b56f-263f-48a1-90f1-2ed930e36b78 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.564229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] The token cache is being invalidate because project c42d9d3ad9c64a768246fed6b93281a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:00.564562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:00.565224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7f6b119b-7b5d-419f-a0aa-25bb6ca00ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:00.566778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 960/1942] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:00 2022] DELETE /identity/v3/projects/c42d9d3ad9c64a768246fed6b93281a4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:00.571968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.577188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.577188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.593053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:00.594406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.608687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.611559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.611559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.611559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.611559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.612820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.627094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 983/1943] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:00.627094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.627357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.656985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 961/1944] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:00.657724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.659141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.671837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293047d2-cb7d-42eb-8484-9766c56bae7d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.676954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53558291-37bc-4974-8be7-ff09cde842eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.677693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53558291-37bc-4974-8be7-ff09cde842eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.677981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53558291-37bc-4974-8be7-ff09cde842eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.678254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53558291-37bc-4974-8be7-ff09cde842eb admin admin] PATH_INFO: `/v3/projects/b9c65645583241288a9808e3a531b3a4/users/25f0ee3f17e642db841c247eca04941a/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.681109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53558291-37bc-4974-8be7-ff09cde842eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c65645583241288a9808e3a531b3a4, user_id=25f0ee3f17e642db841c247eca04941a, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.683803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53558291-37bc-4974-8be7-ff09cde842eb admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.689599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ee057-9e4a-4338-b9f0-f7df996ca8e4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.694810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 962/1945] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:00 2022] PUT /identity/v3/projects/b9c65645583241288a9808e3a531b3a4/users/25f0ee3f17e642db841c247eca04941a/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:00.696714 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a31c-995f-43cc-8654-ae6fe4f95637 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.697033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a31c-995f-43cc-8654-ae6fe4f95637 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.697271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a31c-995f-43cc-8654-ae6fe4f95637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.718281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1030a31c-995f-43cc-8654-ae6fe4f95637 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:00.719512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1030a31c-995f-43cc-8654-ae6fe4f95637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.760774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.761901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.762370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.762737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.765019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.798823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 963/1946] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:00.800434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43936452-5995-47ea-b443-c35dbce58add None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.800799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43936452-5995-47ea-b443-c35dbce58add None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.801173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43936452-5995-47ea-b443-c35dbce58add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.831316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-43936452-5995-47ea-b443-c35dbce58add None None] MFA Rules not processed for user `25f0ee3f17e642db841c247eca04941a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:00.832913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-43936452-5995-47ea-b443-c35dbce58add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.838919 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.840615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.853254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 984/1947] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:00.854202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.865884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5f995-aa48-4e9b-a44d-d9ed5786a92f service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.877103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.877909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.878254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.878574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.882578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:00.882578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.882578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.899511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 985/1948] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:14:00.903869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.918042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7401f80-9fbd-4f7d-a61b-767cb7f42576 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.922117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 964/1949] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:00.933860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43936452-5995-47ea-b443-c35dbce58add None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:00.935443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-43936452-5995-47ea-b443-c35dbce58add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:00.939342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.939989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.940230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.941316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.942000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:00.955560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:00.955713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43936452-5995-47ea-b443-c35dbce58add None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:00.955713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:00.955713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:00.955713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:00.956050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:00.956608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:01.021397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:01.022922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:01.037963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 965/1950] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:01.094365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 986/1951] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:00 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:01.095499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:01.108560 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580d5fb-3c04-4c4d-b1de-4cbcec1a9cbd service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:01.173889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:01.173889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:01.173889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:01.173889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:01.174873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:01.177883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:01.238282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:01.302453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 966/1952] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:01 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:01.314531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:01.324930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd144c5-4586-4c86-b63d-5fbf5f7ec34e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:01.329531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:01.330148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:01.330429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:01.330706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:01.331405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:01.333068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:01.343532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 987/1953] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:01.347459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:01.358330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373d4e7-6559-405d-a5fd-b4047b3ae210 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:01.362937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f19a2-9f52-4be6-b4a9-5ff094a2f978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:01.363540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f19a2-9f52-4be6-b4a9-5ff094a2f978 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:01.363820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f19a2-9f52-4be6-b4a9-5ff094a2f978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:01.364093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f19a2-9f52-4be6-b4a9-5ff094a2f978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:01.364816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f19a2-9f52-4be6-b4a9-5ff094a2f978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:01.366391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f19a2-9f52-4be6-b4a9-5ff094a2f978 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:01.383158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 967/1954] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:01.387562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:01.401067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981e01a-3bbb-48b6-9ae1-20913a703869 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:01.406666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:01.407353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:01.407638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:01.407978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] PATH_INFO: `/v3/projects/3dbfba3e111a4f5c9aece16cd0de9abc/users/270d240bd92e42cea4069a32a07d8ef9/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:01.424094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dbfba3e111a4f5c9aece16cd0de9abc, user_id=270d240bd92e42cea4069a32a07d8ef9, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:01.426949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:01.468201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:01.536811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 988/1955] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:01 2022] PUT /identity/v3/projects/3dbfba3e111a4f5c9aece16cd0de9abc/users/270d240bd92e42cea4069a32a07d8ef9/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:01.547613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622efec8-0b76-4db3-935b-ac52f6559529 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:01.547897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622efec8-0b76-4db3-935b-ac52f6559529 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:01.548173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622efec8-0b76-4db3-935b-ac52f6559529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:01.577619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-622efec8-0b76-4db3-935b-ac52f6559529 None None] MFA Rules not processed for user `270d240bd92e42cea4069a32a07d8ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:01.579038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-622efec8-0b76-4db3-935b-ac52f6559529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:01.644600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 968/1956] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:01.649125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:01.713864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69a00-d28f-4c37-9867-af8e2aa62978 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:01.719036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:01.719700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:01.719979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:01.720257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:01.722262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:01.722929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:01.907443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 989/1957] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:01 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:01.911427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622efec8-0b76-4db3-935b-ac52f6559529 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:01.925638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622efec8-0b76-4db3-935b-ac52f6559529 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:01.997038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:01.997811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:01.998298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:01.998674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:01.999427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:02.002415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:02.149277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 969/1958] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:01 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:02.161815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:02.174701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799231f-84f3-4a04-a2e8-e2674e81db7a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:02.183066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:02.184183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.184657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.185176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.186120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:02.187997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:02.202791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 990/1959] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:02.217696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:02.235306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c097c1-6e2d-4508-8e70-2f919c5a2b51 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:02.247142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:02.247142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.247142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.247142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] PATH_INFO: `/v3/projects/f42a192297c4431f9aa6399413ddcfa0/users/1d3c7a9286a9428784cd1f54f1e294e4/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.263317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42a192297c4431f9aa6399413ddcfa0, user_id=1d3c7a9286a9428784cd1f54f1e294e4, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:02.266440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:02.351158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:02.363700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 970/1960] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:02 2022] PUT /identity/v3/projects/f42a192297c4431f9aa6399413ddcfa0/users/1d3c7a9286a9428784cd1f54f1e294e4/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:02.364811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:02.383602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f9990-de7c-4122-b9a8-90d3d2f7d4e4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:02.457584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f96b1d-65f0-4428-a873-1a9eb25ec287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:02.458497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f96b1d-65f0-4428-a873-1a9eb25ec287 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.458953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f96b1d-65f0-4428-a873-1a9eb25ec287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.459393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f96b1d-65f0-4428-a873-1a9eb25ec287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.460272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f96b1d-65f0-4428-a873-1a9eb25ec287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:02.462733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f96b1d-65f0-4428-a873-1a9eb25ec287 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:02.477409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 991/1961] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:02.487381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:02.502524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8acb4-f4b5-49da-9640-3b29b3e29d3d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:02.509364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:02.510053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.510338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.510635 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] PATH_INFO: `/v3/projects/f42a192297c4431f9aa6399413ddcfa0/users/1d3c7a9286a9428784cd1f54f1e294e4/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.512800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42a192297c4431f9aa6399413ddcfa0, user_id=1d3c7a9286a9428784cd1f54f1e294e4, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:02.517157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:02.532601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 971/1962] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:02 2022] PUT /identity/v3/projects/f42a192297c4431f9aa6399413ddcfa0/users/1d3c7a9286a9428784cd1f54f1e294e4/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:02.543538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd63f6-6518-4f04-9d0d-2118693b5a00 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.544216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd63f6-6518-4f04-9d0d-2118693b5a00 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.544756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd63f6-6518-4f04-9d0d-2118693b5a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.577617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8abd63f6-6518-4f04-9d0d-2118693b5a00 None None] MFA Rules not processed for user `1d3c7a9286a9428784cd1f54f1e294e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:02.579059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8abd63f6-6518-4f04-9d0d-2118693b5a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:02.657420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 992/1963] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:02.669205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:02.732082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891fe29-0647-46f1-8d31-5468fc536be7 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:02.738290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f26b3-6547-47c5-bc81-0230781f40df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:02.738290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f26b3-6547-47c5-bc81-0230781f40df service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.738290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f26b3-6547-47c5-bc81-0230781f40df service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.738290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f26b3-6547-47c5-bc81-0230781f40df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.749956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f26b3-6547-47c5-bc81-0230781f40df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:02.751484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f26b3-6547-47c5-bc81-0230781f40df service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:02.762886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 972/1964] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:02.819521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05517f-b75e-41ad-87fa-3621bf87703a None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.819827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05517f-b75e-41ad-87fa-3621bf87703a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.820089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05517f-b75e-41ad-87fa-3621bf87703a None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.822136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 993/1965] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:14:02.840141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ee829-bbc5-4c1f-96fd-a2d79c1e948b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:02.840544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ee829-bbc5-4c1f-96fd-a2d79c1e948b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:02.840823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ee829-bbc5-4c1f-96fd-a2d79c1e948b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:02.859931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-377ee829-bbc5-4c1f-96fd-a2d79c1e948b None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:02.861184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-377ee829-bbc5-4c1f-96fd-a2d79c1e948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:02.948718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 973/1966] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:14:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:02.970881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05517f-b75e-41ad-87fa-3621bf87703a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:02.972746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8d05517f-b75e-41ad-87fa-3621bf87703a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:03.010980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05517f-b75e-41ad-87fa-3621bf87703a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:03.018701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da311d-1228-4919-812b-fa62ff26fb31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:03.019696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da311d-1228-4919-812b-fa62ff26fb31 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:03.024494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da311d-1228-4919-812b-fa62ff26fb31 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:03.025401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da311d-1228-4919-812b-fa62ff26fb31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:03.037393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da311d-1228-4919-812b-fa62ff26fb31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:03.040791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da311d-1228-4919-812b-fa62ff26fb31 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:03.054177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 994/1967] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:03.177006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:03.307986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:03.495439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ee829-bbc5-4c1f-96fd-a2d79c1e948b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:03.558155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ee829-bbc5-4c1f-96fd-a2d79c1e948b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:03.562856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:03.563503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:03.563792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:03.564070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] PATH_INFO: `/v3/users/270d240bd92e42cea4069a32a07d8ef9` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:03.565311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=270d240bd92e42cea4069a32a07d8ef9)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:03.567243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:03.772784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '270d240bd92e42cea4069a32a07d8ef9'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:03.774077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '270d240bd92e42cea4069a32a07d8ef9'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:03.861726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '270d240bd92e42cea4069a32a07d8ef9'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:03.874838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '270d240bd92e42cea4069a32a07d8ef9'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:03.883506 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '270d240bd92e42cea4069a32a07d8ef9'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:03.888212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 974/1968] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:03 2022] DELETE /identity/v3/users/270d240bd92e42cea4069a32a07d8ef9 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:03.892632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:03.893108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:03.893200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:03.912629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:03.913902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:03.980894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 995/1969] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:03.987612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:03.989139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:04.046074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0b9f6-2f5d-402c-8225-2258d3e95f77 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:04.049591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:04.050200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:04.050476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:04.050747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:04.062777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:04.064225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:04.074375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 975/1970] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:04.106549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:04.353710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:04.410855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:04.516130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:04.516816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:04.537850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f585-3e5f-4f36-9e99-368ed33bb987 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:04.544735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:04.545412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:04.545688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:04.546103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] PATH_INFO: `/v3/projects/3dbfba3e111a4f5c9aece16cd0de9abc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:04.547366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dbfba3e111a4f5c9aece16cd0de9abc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:04.549597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:04.685986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dbfba3e111a4f5c9aece16cd0de9abc', 'request_id': 'req-9f4257be-8eb1-493a-be51-c5f5263c294d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:04.692223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dbfba3e111a4f5c9aece16cd0de9abc', 'request_id': 'req-9f4257be-8eb1-493a-be51-c5f5263c294d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:04.698337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dbfba3e111a4f5c9aece16cd0de9abc', 'request_id': 'req-9f4257be-8eb1-493a-be51-c5f5263c294d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:04.786718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] The token cache is being invalidate because project 3dbfba3e111a4f5c9aece16cd0de9abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:04.787178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:04.787885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:04.790173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 996/1971] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:04 2022] DELETE /identity/v3/projects/3dbfba3e111a4f5c9aece16cd0de9abc => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:04.794234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:04.805201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d4e4c-a927-4b97-9c74-b17e3423dd9c service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:04.845518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:04.864007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:04.864414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:04.864655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:04.864930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] PATH_INFO: `/v3/users/1d3c7a9286a9428784cd1f54f1e294e4` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:04.866143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d3c7a9286a9428784cd1f54f1e294e4)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:04.869759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:04.953127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:04.999775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:05.000767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d3c7a9286a9428784cd1f54f1e294e4'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.001441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d3c7a9286a9428784cd1f54f1e294e4'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.056465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:05.073066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d3c7a9286a9428784cd1f54f1e294e4'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.085617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d3c7a9286a9428784cd1f54f1e294e4'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.093132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c4c9706e-632c-4cd0-91b7-76f03dcd7cbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d3c7a9286a9428784cd1f54f1e294e4'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.099177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 976/1972] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:04 2022] DELETE /identity/v3/users/1d3c7a9286a9428784cd1f54f1e294e4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:05.386686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:05.411969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:05.561166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:05.562911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:05.578457 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4257be-8eb1-493a-be51-c5f5263c294d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:05.634041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:05.647828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:05.649140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:05.649426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:05.649704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] PATH_INFO: `/v3/projects/f42a192297c4431f9aa6399413ddcfa0` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:05.650876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42a192297c4431f9aa6399413ddcfa0)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:05.652869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:05.767383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:05.797099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42a192297c4431f9aa6399413ddcfa0', 'request_id': 'req-afac5c30-4c40-455c-83e0-3f0f1f515e49'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.804426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42a192297c4431f9aa6399413ddcfa0', 'request_id': 'req-afac5c30-4c40-455c-83e0-3f0f1f515e49'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.814793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42a192297c4431f9aa6399413ddcfa0', 'request_id': 'req-afac5c30-4c40-455c-83e0-3f0f1f515e49'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.921900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] The token cache is being invalidate because project f42a192297c4431f9aa6399413ddcfa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:05.922528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.923435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-afac5c30-4c40-455c-83e0-3f0f1f515e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:05.925251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 997/1973] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:05 2022] DELETE /identity/v3/projects/f42a192297c4431f9aa6399413ddcfa0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:05.930425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f934f5-f5bd-4566-a2c9-2ca975697649 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:05.931191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f934f5-f5bd-4566-a2c9-2ca975697649 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:05.931191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f934f5-f5bd-4566-a2c9-2ca975697649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:05.952100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-79f934f5-f5bd-4566-a2c9-2ca975697649 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:05.953413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-79f934f5-f5bd-4566-a2c9-2ca975697649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:06.019448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 977/1974] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:06.028316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa7c8-32dc-40e9-bef9-8deea1d14576 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.028316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa7c8-32dc-40e9-bef9-8deea1d14576 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.028916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aa7c8-32dc-40e9-bef9-8deea1d14576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.077679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b59aa7c8-32dc-40e9-bef9-8deea1d14576 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:06.079119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b59aa7c8-32dc-40e9-bef9-8deea1d14576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:06.089798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:06.093612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 998/1975] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:06.103351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f934f5-f5bd-4566-a2c9-2ca975697649 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.114287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f934f5-f5bd-4566-a2c9-2ca975697649 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.123580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.123580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.123580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.123950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.124466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:06.124778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.126434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:06.138925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 978/1976] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:14:06.143241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aa7c8-32dc-40e9-bef9-8deea1d14576 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.153007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aa7c8-32dc-40e9-bef9-8deea1d14576 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.158050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.158668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.158943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.159219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.160006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.162002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:06.301838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 999/1977] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:06 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:06.307524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.317581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f601c-0b51-4350-9603-b89bab448332 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.385968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.386607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.386913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.387159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.387846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.389920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:06.489257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 979/1978] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:06 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:06.494385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.504522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60221ec-031a-4b94-9712-1af7ca41fed9 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.509954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.510569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.510854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.511131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.511832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.513467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:06.526970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1000/1979] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:06.532253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.544281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb8f9f-29c1-476e-9a19-4d0920b67b55 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.547528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d79054-1b0e-4735-9ddf-ab68afaf505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.548148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d79054-1b0e-4735-9ddf-ab68afaf505e admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.548458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d79054-1b0e-4735-9ddf-ab68afaf505e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.548736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d79054-1b0e-4735-9ddf-ab68afaf505e admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.549438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d79054-1b0e-4735-9ddf-ab68afaf505e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.551061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d79054-1b0e-4735-9ddf-ab68afaf505e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:06.563972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 980/1980] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:06.568257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.579293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589d047-5c58-4b13-96bd-99cd0e5a8066 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.587704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.588494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.588850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.589204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] PATH_INFO: `/v3/projects/eeffbfd0989d4f34a555e009eb55f186/users/a98d2000e83c4537a95b2e2417a0fc84/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.614636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeffbfd0989d4f34a555e009eb55f186, user_id=a98d2000e83c4537a95b2e2417a0fc84, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.614636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:06.631832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:06.714339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1001/1981] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:06 2022] PUT /identity/v3/projects/eeffbfd0989d4f34a555e009eb55f186/users/a98d2000e83c4537a95b2e2417a0fc84/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:06.729402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414919fb-9945-43cf-aba7-860a541efad8 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.730112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414919fb-9945-43cf-aba7-860a541efad8 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.730547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414919fb-9945-43cf-aba7-860a541efad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.734961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:06.773203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-414919fb-9945-43cf-aba7-860a541efad8 None None] MFA Rules not processed for user `a98d2000e83c4537a95b2e2417a0fc84`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:06.773203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-414919fb-9945-43cf-aba7-860a541efad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:06.774778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.776505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:06.860527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd1342-289c-41f8-b0b5-84ce0e4e3fc4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.867527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.868672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.872522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 981/1982] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:06.873157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.873658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] PATH_INFO: `/v3/users/8c32d86f55a54b00a912caa740a20f52` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.873988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414919fb-9945-43cf-aba7-860a541efad8 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.875361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c32d86f55a54b00a912caa740a20f52)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.877942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:06.886946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414919fb-9945-43cf-aba7-860a541efad8 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.904656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.905379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.905854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.906334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.910457 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:06.913371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.032809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 982/1983] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:06 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:07.033770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.106574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650992b-3bc5-4ab9-be1a-2bc8204070b4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:07.111679 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:07.112653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.112879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.114276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.114276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.116605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.138228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c32d86f55a54b00a912caa740a20f52'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:07.139367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c32d86f55a54b00a912caa740a20f52'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:07.221978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c32d86f55a54b00a912caa740a20f52'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:07.228569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 983/1984] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:07 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:07.233464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.237738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c32d86f55a54b00a912caa740a20f52'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:07.239183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401f1e6c-865b-43cc-b5d0-70b780758144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c32d86f55a54b00a912caa740a20f52'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:07.251175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0515-73b7-4673-af85-faf9040a9d32 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:07.253934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1002/1985] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:06 2022] DELETE /identity/v3/users/8c32d86f55a54b00a912caa740a20f52 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:07.259113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.259404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.259685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.281701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:07.282951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:07.370501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:07.371307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.371589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.371864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.372602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.374199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.387177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 984/1986] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:07.391078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.394266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:07.401614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1003/1987] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:07.414241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.415220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:07.419528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03674e2-5fcf-4db3-b95f-5982242fd7d8 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:07.434887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:07.435821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.436255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.436581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] PATH_INFO: `/v3/projects/06715afdf61d42b5b3b71c13826440a8/users/36b5d728574a4f31b336275b24ed2a14/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.453606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06715afdf61d42b5b3b71c13826440a8, user_id=36b5d728574a4f31b336275b24ed2a14, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.456461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.482652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896508fa-ebb4-423b-96f4-3e90571a6ab9 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:07.486916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:07.487556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.487840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.488115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.498610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.500704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.512456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1004/1988] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:07.554026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 985/1989] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:07 2022] PUT /identity/v3/projects/06715afdf61d42b5b3b71c13826440a8/users/36b5d728574a4f31b336275b24ed2a14/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:07.558067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.568752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1814f0e-1ab9-4c40-a78c-80255798333d service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:07.637624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d43324-d5c8-4ef5-8825-ffdc36e0eee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:07.638265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43324-d5c8-4ef5-8825-ffdc36e0eee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.638549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43324-d5c8-4ef5-8825-ffdc36e0eee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.638825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43324-d5c8-4ef5-8825-ffdc36e0eee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.639526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d43324-d5c8-4ef5-8825-ffdc36e0eee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.641318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d43324-d5c8-4ef5-8825-ffdc36e0eee4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.652645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1005/1990] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:07.656960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.667966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd764b-514e-4c03-9d94-69b60290b984 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:07.673336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:07.674182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.674345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.674646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] PATH_INFO: `/v3/projects/06715afdf61d42b5b3b71c13826440a8/users/36b5d728574a4f31b336275b24ed2a14/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.676659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06715afdf61d42b5b3b71c13826440a8, user_id=36b5d728574a4f31b336275b24ed2a14, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.679315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.690420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 986/1991] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:07 2022] PUT /identity/v3/projects/06715afdf61d42b5b3b71c13826440a8/users/36b5d728574a4f31b336275b24ed2a14/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:07.702292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.702624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.707586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.712662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:07.734359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] MFA Rules not processed for user `36b5d728574a4f31b336275b24ed2a14`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:07.735577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:07.780723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:07.813099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1006/1992] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:07.825780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.827633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:07.887777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c4f6a-bbd8-48f9-850d-1957ab706311 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:07.892411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:07.893107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:07.893414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:07.893660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:07.903144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.904665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.914153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 987/1993] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:07.937491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:07.939030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:08.012208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d1b8-3ed2-4466-b8c6-53802d0db931 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:08.017373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:08.018118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:08.018413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:08.018672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] PATH_INFO: `/v3/projects/6c503ba498444f59b55f7127316a5fbf` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:08.019851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c503ba498444f59b55f7127316a5fbf)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:08.021781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:08.097282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:08.118002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:08.118716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c503ba498444f59b55f7127316a5fbf', 'request_id': 'req-04a2409f-6cf0-4d5e-87c1-df669762208d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.125640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c503ba498444f59b55f7127316a5fbf', 'request_id': 'req-04a2409f-6cf0-4d5e-87c1-df669762208d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.127630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9e23a-f61f-45a0-847e-32f2e1ebece5 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:08.131572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c503ba498444f59b55f7127316a5fbf', 'request_id': 'req-04a2409f-6cf0-4d5e-87c1-df669762208d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.188247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:08.189003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:08.189379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:08.189725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:08.191732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:08.240867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] The token cache is being invalidate because project 6c503ba498444f59b55f7127316a5fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:08.242999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.245246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.252679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1007/1994] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:07 2022] DELETE /identity/v3/projects/6c503ba498444f59b55f7127316a5fbf => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:08.254603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:08.258234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:08.269850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:08.294750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:08.296250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:08.306661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 988/1995] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:08.341702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2409f-6cf0-4d5e-87c1-df669762208d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:08.348491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:08.349193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:08.349750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:08.350755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] PATH_INFO: `/v3/users/2aad885ed9c945fe8b21e1db3db51891` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:08.354125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aad885ed9c945fe8b21e1db3db51891)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:08.357637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:08.672534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aad885ed9c945fe8b21e1db3db51891'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.678649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aad885ed9c945fe8b21e1db3db51891'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.755122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aad885ed9c945fe8b21e1db3db51891'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.776671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aad885ed9c945fe8b21e1db3db51891'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.777642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aad885ed9c945fe8b21e1db3db51891'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:08.795693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1008/1996] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:08 2022] DELETE /identity/v3/users/2aad885ed9c945fe8b21e1db3db51891 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:09.263373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:09.271355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:09.293897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f25d5e-0ead-4f33-a7d8-68a60322e331 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:09.367523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:09.368265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:09.368644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:09.369271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] PATH_INFO: `/v3/projects/2e7a617f69ce4801bcef8b79a90d3342` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:09.370590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7a617f69ce4801bcef8b79a90d3342)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:09.372689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:09.506643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:09.516080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:09.522893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7a617f69ce4801bcef8b79a90d3342', 'request_id': 'req-3dd56b5b-aff0-4bdd-ab0c-2441285effec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.528665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b661bf-5bf8-4b33-9e7b-5e6d043ff430 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:09.533711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7a617f69ce4801bcef8b79a90d3342', 'request_id': 'req-3dd56b5b-aff0-4bdd-ab0c-2441285effec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.540242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7a617f69ce4801bcef8b79a90d3342', 'request_id': 'req-3dd56b5b-aff0-4bdd-ab0c-2441285effec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.586395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:09.587052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:09.588761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:09.588761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] PATH_INFO: `/v3/users/a98d2000e83c4537a95b2e2417a0fc84` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:09.588950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98d2000e83c4537a95b2e2417a0fc84)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:09.590802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:09.628232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:09.635568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] The token cache is being invalidate because project 2e7a617f69ce4801bcef8b79a90d3342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:09.635897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.636569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3dd56b5b-aff0-4bdd-ab0c-2441285effec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.638105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 989/1997] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:09 2022] DELETE /identity/v3/projects/2e7a617f69ce4801bcef8b79a90d3342 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:09.642974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f95115-8344-4e3e-89dd-788fa52913c2 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:09.643557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f95115-8344-4e3e-89dd-788fa52913c2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:09.646215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f95115-8344-4e3e-89dd-788fa52913c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:09.669619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a1f95115-8344-4e3e-89dd-788fa52913c2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:09.671429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a1f95115-8344-4e3e-89dd-788fa52913c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:09.681367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 990/1998] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:09.689442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7c84-6c62-484b-8aaa-6a3fdb823850 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:09.689954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7c84-6c62-484b-8aaa-6a3fdb823850 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:09.690529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7c84-6c62-484b-8aaa-6a3fdb823850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:09.721258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-419a7c84-6c62-484b-8aaa-6a3fdb823850 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:09.722626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-419a7c84-6c62-484b-8aaa-6a3fdb823850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:09.732270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 991/1999] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:09.737247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7c84-6c62-484b-8aaa-6a3fdb823850 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:09.753698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7c84-6c62-484b-8aaa-6a3fdb823850 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:09.759462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:09.760101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:09.760437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:09.760690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:09.761404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:09.761714 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:09.763418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:09.776369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 992/2000] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:14:09.780665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:09.790852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ffdaa-4a17-4c05-aa51-f9ba394527c4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:09.796507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:09.797448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:09.797816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:09.798985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:09.799702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:09.801752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:09.807923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98d2000e83c4537a95b2e2417a0fc84'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.808764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98d2000e83c4537a95b2e2417a0fc84'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.885074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98d2000e83c4537a95b2e2417a0fc84'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.906632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98d2000e83c4537a95b2e2417a0fc84'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.907674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 993/2001] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:09 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:09.908531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0aeeed11-8f66-4a93-ae6f-b411873e7360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98d2000e83c4537a95b2e2417a0fc84'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:09.911743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:09.917669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:09.938529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1009/2002] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:09 2022] DELETE /identity/v3/users/a98d2000e83c4537a95b2e2417a0fc84 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:09.940089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:09.940816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:09.941266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:09.962827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:09.964056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:09.994822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90cb32-3e6a-4a60-8028-4e8793d6de73 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.049018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1010/2003] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:10.057009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.059333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:10.075258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.076169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.076169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.076521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.077197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.079825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.128287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b55e-6fda-4d49-ae52-483ab52a86d3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.132146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.133436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.133826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.134183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.143880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.145433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.155287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1011/2004] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:10.205844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 994/2005] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:09 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:10.210301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.219951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb991b-5036-4395-9fa9-96947e6957b4 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.225267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.225889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.226202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.226478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.227175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.228833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.242315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1012/2006] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:10.243779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.254542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28810a-a842-4cb7-af2d-9560039a1342 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.259916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c054062-0dac-4eab-9e86-7930c309fecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.260574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c054062-0dac-4eab-9e86-7930c309fecd admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.260855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c054062-0dac-4eab-9e86-7930c309fecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.261128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c054062-0dac-4eab-9e86-7930c309fecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.261832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c054062-0dac-4eab-9e86-7930c309fecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.263423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c054062-0dac-4eab-9e86-7930c309fecd admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.273513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 995/2007] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:10.277766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.287610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6276bad-b894-4bec-a0c7-a0b40255c1e9 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.292301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.293015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.293333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.293616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] PATH_INFO: `/v3/projects/3f76109845c547b8a2253df709c91826/users/9b149e74681044959e6eba750d9a26cc/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.315549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f76109845c547b8a2253df709c91826, user_id=9b149e74681044959e6eba750d9a26cc, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.318318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.423211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1013/2008] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:10 2022] PUT /identity/v3/projects/3f76109845c547b8a2253df709c91826/users/9b149e74681044959e6eba750d9a26cc/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:10.435151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.435470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.435762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.490253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] MFA Rules not processed for user `9b149e74681044959e6eba750d9a26cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:10.492886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:10.565650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 996/2009] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:10.570982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.615881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.617692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:10.641095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a721ea-e527-4757-b137-8adcc7ffe8c1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.646060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.646681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.646958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.647228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.647920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.649932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.686893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f544cd8-bbd3-4e1d-81bb-99365d8e6270 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.691926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.692587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.692870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.693170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] PATH_INFO: `/v3/projects/eeffbfd0989d4f34a555e009eb55f186` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.694376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeffbfd0989d4f34a555e009eb55f186)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.696826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.719356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:10.753401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1014/2010] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:10 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:10.757799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.816725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f13a94-76ab-4740-ab0e-61dcb9dff20d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.821878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.822472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.822750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.823014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.823672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.825686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.832856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeffbfd0989d4f34a555e009eb55f186', 'request_id': 'req-43686fd5-8247-41df-810a-29106d19ec69'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:10.839647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeffbfd0989d4f34a555e009eb55f186', 'request_id': 'req-43686fd5-8247-41df-810a-29106d19ec69'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:10.845204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeffbfd0989d4f34a555e009eb55f186', 'request_id': 'req-43686fd5-8247-41df-810a-29106d19ec69'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:10.907374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1015/2011] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:10 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:10.911027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.920372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4eccd-65f7-4071-a06e-f197a9d2be3e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:10.936959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] The token cache is being invalidate because project eeffbfd0989d4f34a555e009eb55f186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:10.937906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:10.938871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:10.940345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 997/2012] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:10 2022] DELETE /identity/v3/projects/eeffbfd0989d4f34a555e009eb55f186 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:10.944130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:10.946480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:10.980633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:10.981265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:10.981552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:10.981858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:10.982568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:10.984162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:10.998575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1016/2013] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:10.999494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.001910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:11.017394 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28572159-7b59-4f2a-bde1-aafafa33fd84 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.017807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43686fd5-8247-41df-810a-29106d19ec69 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.022787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.023417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.023696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.023967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.024093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.024657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.024774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.024955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.025218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] PATH_INFO: `/v3/users/36b5d728574a4f31b336275b24ed2a14` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.026393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.026621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b5d728574a4f31b336275b24ed2a14)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.028699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.043561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1017/2014] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:11.048031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.058853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b058b9-bdd2-4036-b0df-28a73a3bc5d5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.063661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9b985-be1d-4819-b3ca-10604c7f65c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.064325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b985-be1d-4819-b3ca-10604c7f65c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.064642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b985-be1d-4819-b3ca-10604c7f65c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.064923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9b985-be1d-4819-b3ca-10604c7f65c6 admin admin] PATH_INFO: `/v3/projects/ff26d49c401c490d9a42f0329a743faa/users/dca962d7ae3341feb5a53a45cf38ddff/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.080638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9b985-be1d-4819-b3ca-10604c7f65c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff26d49c401c490d9a42f0329a743faa, user_id=dca962d7ae3341feb5a53a45cf38ddff, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.083285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9b985-be1d-4819-b3ca-10604c7f65c6 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.168869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1018/2015] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:11 2022] PUT /identity/v3/projects/ff26d49c401c490d9a42f0329a743faa/users/dca962d7ae3341feb5a53a45cf38ddff/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:11.180995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76339c-224f-456e-9f83-42b6b90a3346 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.180995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76339c-224f-456e-9f83-42b6b90a3346 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.181161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76339c-224f-456e-9f83-42b6b90a3346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.209670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-ef76339c-224f-456e-9f83-42b6b90a3346 None None] MFA Rules not processed for user `dca962d7ae3341feb5a53a45cf38ddff`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:11.210884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ef76339c-224f-456e-9f83-42b6b90a3346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:11.229969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b5d728574a4f31b336275b24ed2a14'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.230654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b5d728574a4f31b336275b24ed2a14'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.279133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1019/2016] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:11.283651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76339c-224f-456e-9f83-42b6b90a3346 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.301649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b5d728574a4f31b336275b24ed2a14'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.314955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b5d728574a4f31b336275b24ed2a14'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.322357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b5d728574a4f31b336275b24ed2a14'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.328902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 998/2017] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:10 2022] DELETE /identity/v3/users/36b5d728574a4f31b336275b24ed2a14 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:11.354203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76339c-224f-456e-9f83-42b6b90a3346 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.413290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.414556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.414840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.415115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.415784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.417778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.509527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1020/2018] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:11 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:11.513259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.517601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:11.537760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e1fb1-c65b-45de-acdb-b5e15fc88a94 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.610436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.611201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.611380 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.611629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.612398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.614520 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.622660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:11.721624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 999/2019] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:11 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:11.722882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.733697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b0f0e-de67-4ca8-bec0-cac21a322acb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.738534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.739242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.739625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.739789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.741351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.742254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.745857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.747939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:11.762743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3bf08-878d-475a-9625-eabb73e98e6c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.779193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1021/2020] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:11.779820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.780631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.780812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.781143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] PATH_INFO: `/v3/projects/06715afdf61d42b5b3b71c13826440a8` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.782329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06715afdf61d42b5b3b71c13826440a8)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.783635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.784249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.795724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e158af-44ee-43b1-b053-fed42e785c24 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:11.800986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:11.801651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:11.801925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:11.806949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] PATH_INFO: `/v3/projects/a3a31663bb7b45879cde0bcb8812ea1d/users/236238eb772d48b6a75f301981a1a597/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:11.816740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a31663bb7b45879cde0bcb8812ea1d, user_id=236238eb772d48b6a75f301981a1a597, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:11.819352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:11.826216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:11.920167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1022/2021] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:11 2022] PUT /identity/v3/projects/a3a31663bb7b45879cde0bcb8812ea1d/users/236238eb772d48b6a75f301981a1a597/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:11.923914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:11.929802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06715afdf61d42b5b3b71c13826440a8', 'request_id': 'req-074a31a1-6915-4050-89bd-34f758dd4f83'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.938460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06715afdf61d42b5b3b71c13826440a8', 'request_id': 'req-074a31a1-6915-4050-89bd-34f758dd4f83'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.944721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06715afdf61d42b5b3b71c13826440a8', 'request_id': 'req-074a31a1-6915-4050-89bd-34f758dd4f83'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:11.987160 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecc83f-28a2-4eb6-8663-59a6d28b1f2d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.027147 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] The token cache is being invalidate because project 06715afdf61d42b5b3b71c13826440a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:12.027416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:12.028057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-074a31a1-6915-4050-89bd-34f758dd4f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:12.029625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1000/2022] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:11 2022] DELETE /identity/v3/projects/06715afdf61d42b5b3b71c13826440a8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:12.034340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d267-fa25-4475-a11a-3695b59bf552 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.034626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d267-fa25-4475-a11a-3695b59bf552 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.034899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d267-fa25-4475-a11a-3695b59bf552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.046197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.046828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.047110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.047382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.048124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.049754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.054092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-23a1d267-fa25-4475-a11a-3695b59bf552 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:12.055297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-23a1d267-fa25-4475-a11a-3695b59bf552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.062630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1023/2023] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:12.063648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.064482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1001/2024] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:12.065324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.072645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74feeb1-1d82-40a2-86a6-6e4b5e41f226 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.072935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74feeb1-1d82-40a2-86a6-6e4b5e41f226 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.073241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74feeb1-1d82-40a2-86a6-6e4b5e41f226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.078179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c67d16-6abd-446e-910e-f828d3875c9b admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.083026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de9f35-b926-4b30-ad6c-3ecf410e62c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.083701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de9f35-b926-4b30-ad6c-3ecf410e62c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.083985 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de9f35-b926-4b30-ad6c-3ecf410e62c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.084263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de9f35-b926-4b30-ad6c-3ecf410e62c0 admin admin] PATH_INFO: `/v3/projects/a3a31663bb7b45879cde0bcb8812ea1d/users/236238eb772d48b6a75f301981a1a597/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.086951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de9f35-b926-4b30-ad6c-3ecf410e62c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3a31663bb7b45879cde0bcb8812ea1d, user_id=236238eb772d48b6a75f301981a1a597, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.089772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de9f35-b926-4b30-ad6c-3ecf410e62c0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.093124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b74feeb1-1d82-40a2-86a6-6e4b5e41f226 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:12.094416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b74feeb1-1d82-40a2-86a6-6e4b5e41f226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.100233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1024/2025] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:12 2022] PUT /identity/v3/projects/a3a31663bb7b45879cde0bcb8812ea1d/users/236238eb772d48b6a75f301981a1a597/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:12.110690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.110977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.111256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.142093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] MFA Rules not processed for user `236238eb772d48b6a75f301981a1a597`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:12.143338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.157554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1002/2026] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:12.161637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74feeb1-1d82-40a2-86a6-6e4b5e41f226 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.171123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74feeb1-1d82-40a2-86a6-6e4b5e41f226 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.175896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.176628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.176987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.177367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.178152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:12.178532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.180263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.193113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1003/2027] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:14:12.198176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.209230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdca1c-99d9-4243-9327-db0ca475d7d8 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.214730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.215341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.215784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.216214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.216943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.217463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1025/2028] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:12.222456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.232237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.239252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.294577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07733b8-e256-4742-9c68-440bb9a9b939 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.298232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e998-9110-4404-b546-60492316650b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.298842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e998-9110-4404-b546-60492316650b service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.299119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e998-9110-4404-b546-60492316650b service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.299389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e998-9110-4404-b546-60492316650b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.301187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e886e998-9110-4404-b546-60492316650b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.314435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1004/2029] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:12 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:12.318499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.361376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:12.363655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886e998-9110-4404-b546-60492316650b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.367279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886e998-9110-4404-b546-60492316650b service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.377132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1026/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:12.386931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa3327-6de1-4d2d-b0ad-4fb60418ea2e admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.397082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.397727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.398052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.398401 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.399096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.401111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.425620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:12.478642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1005/2031] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:12 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:12.484074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e998-9110-4404-b546-60492316650b service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.497037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e998-9110-4404-b546-60492316650b service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.502053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.504355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.504355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.504355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.504355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.505937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.520565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1027/2032] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:12.522053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.533865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918948c-a9cb-4aba-bc01-6e41e69fe8e0 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.541122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2354108-840e-420a-bc21-d375c91dfcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.541787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2354108-840e-420a-bc21-d375c91dfcda admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.546119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2354108-840e-420a-bc21-d375c91dfcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.546119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2354108-840e-420a-bc21-d375c91dfcda admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.546119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2354108-840e-420a-bc21-d375c91dfcda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.546119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2354108-840e-420a-bc21-d375c91dfcda admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.555837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1006/2033] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:12.559931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.570723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67fc1-b0b1-467f-ae3f-7e9c046a3b68 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.577439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.580727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.580727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.580727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] PATH_INFO: `/v3/projects/235700c5cad649bba7cad6c80b9d7633/users/2ffffe97131d483cb55f3eb6bb694840/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.598887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=235700c5cad649bba7cad6c80b9d7633, user_id=2ffffe97131d483cb55f3eb6bb694840, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.602152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.703704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1028/2034] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:12 2022] PUT /identity/v3/projects/235700c5cad649bba7cad6c80b9d7633/users/2ffffe97131d483cb55f3eb6bb694840/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:12.714153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd160874-38ea-4036-b039-faca9c8c37b2 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.714532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd160874-38ea-4036-b039-faca9c8c37b2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.714867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd160874-38ea-4036-b039-faca9c8c37b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.743535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-dd160874-38ea-4036-b039-faca9c8c37b2 None None] MFA Rules not processed for user `2ffffe97131d483cb55f3eb6bb694840`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:12.745093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dd160874-38ea-4036-b039-faca9c8c37b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.822820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1007/2035] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:12.834243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:12.844653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:12.910975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b4046-95d0-4eae-91fe-b8fb9394b68b admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:12.915435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:12.916089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:12.916527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:12.916839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:12.925787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:12.927222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:12.937085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1029/2036] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:12.947437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:13.027260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:13.228218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:13.719856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:13.747316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:13.756282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd160874-38ea-4036-b039-faca9c8c37b2 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:13.766280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd160874-38ea-4036-b039-faca9c8c37b2 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:13.769820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:13.770425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:13.770706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:13.770977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:13.772718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:13.836501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:13.837934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:13.848154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1008/2037] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:13.907337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:13.953681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.124665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.147716 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.267454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.444806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.604800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.763094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.947255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:14.970095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.015674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.113281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.228259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.273676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.415170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.437401 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.518822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.593410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.649588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.890266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:15.913992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:20.775997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:20.784604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1e38b-1f54-4453-8aa1-e21875c34900 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:20.788421 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:20.789019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:20.789294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:20.789564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:20.791642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:20.868496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:20.870174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:20.883873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1030/2038] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:24.290371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:24.298828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb9e6c-9302-42ec-9c30-f7f995128c53 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:24.302531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10e78f-03c9-4015-bfbc-3dcc8bdb2fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:24.303138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10e78f-03c9-4015-bfbc-3dcc8bdb2fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:24.303421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10e78f-03c9-4015-bfbc-3dcc8bdb2fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:24.303702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10e78f-03c9-4015-bfbc-3dcc8bdb2fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:24.364010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10e78f-03c9-4015-bfbc-3dcc8bdb2fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:24.365615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10e78f-03c9-4015-bfbc-3dcc8bdb2fb9 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:24.376999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1009/2039] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:25.161675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:25.251591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:25.256570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15314749-ea5a-426f-aaf5-e159e9936918 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:25.263119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:25.264769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:25.265782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:25.266076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] PATH_INFO: `/v3/users/2ffffe97131d483cb55f3eb6bb694840` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:25.268090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffffe97131d483cb55f3eb6bb694840)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:25.270672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:25.526027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffffe97131d483cb55f3eb6bb694840'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:25.536437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffffe97131d483cb55f3eb6bb694840'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:25.615894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffffe97131d483cb55f3eb6bb694840'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:25.632794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffffe97131d483cb55f3eb6bb694840'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:25.633676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffffe97131d483cb55f3eb6bb694840'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:25.649691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1031/2040] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:25 2022] DELETE /identity/v3/users/2ffffe97131d483cb55f3eb6bb694840 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:25.652813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:25.653323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:25.653509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:25.677289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:25.681348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:25.699005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:25.751843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1010/2041] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:25.764217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:25.765748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:25.827650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869615d-53f5-47db-b43d-f7ca1875e357 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:25.831388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fa5c1-ebef-400a-9708-50d8600f4717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:25.832049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fa5c1-ebef-400a-9708-50d8600f4717 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:25.832329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fa5c1-ebef-400a-9708-50d8600f4717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:25.832641 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fa5c1-ebef-400a-9708-50d8600f4717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:25.842513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fa5c1-ebef-400a-9708-50d8600f4717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:25.847899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533fa5c1-ebef-400a-9708-50d8600f4717 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:25.860680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1032/2042] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:26.422756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:26.424191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:26.449808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ce5d4-f069-4d45-895e-6eff79c0f2f6 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:26.472011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:26.473280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:26.473767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:26.474265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] PATH_INFO: `/v3/projects/235700c5cad649bba7cad6c80b9d7633` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:26.477130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=235700c5cad649bba7cad6c80b9d7633)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:26.478985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:26.635412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235700c5cad649bba7cad6c80b9d7633', 'request_id': 'req-dd681e5e-66b5-4876-b861-f383f2b4b43f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:26.641540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235700c5cad649bba7cad6c80b9d7633', 'request_id': 'req-dd681e5e-66b5-4876-b861-f383f2b4b43f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:26.648161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235700c5cad649bba7cad6c80b9d7633', 'request_id': 'req-dd681e5e-66b5-4876-b861-f383f2b4b43f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:26.766899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] The token cache is being invalidate because project 235700c5cad649bba7cad6c80b9d7633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:26.768036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:26.768729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-dd681e5e-66b5-4876-b861-f383f2b4b43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:26.770410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1011/2043] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:26 2022] DELETE /identity/v3/projects/235700c5cad649bba7cad6c80b9d7633 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:26.777070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f73b-6c3e-4085-93e5-e24551b07749 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:26.777878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f73b-6c3e-4085-93e5-e24551b07749 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:26.778354 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f73b-6c3e-4085-93e5-e24551b07749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:26.800675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-38b2f73b-6c3e-4085-93e5-e24551b07749 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:26.801891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-38b2f73b-6c3e-4085-93e5-e24551b07749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:26.889392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1033/2044] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:26.900559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00cdfe-14a7-4890-98a6-35e00445356c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:26.900559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00cdfe-14a7-4890-98a6-35e00445356c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:26.900559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00cdfe-14a7-4890-98a6-35e00445356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:26.923798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-8c00cdfe-14a7-4890-98a6-35e00445356c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:26.925064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8c00cdfe-14a7-4890-98a6-35e00445356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:26.934479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1012/2045] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:26.940080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f73b-6c3e-4085-93e5-e24551b07749 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:26.952533 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f73b-6c3e-4085-93e5-e24551b07749 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:26.957535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:26.958169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:26.958446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:26.958720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:26.959420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:26.959747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:26.961407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:26.973388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1034/2046] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:14:26.977856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00cdfe-14a7-4890-98a6-35e00445356c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:26.987574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00cdfe-14a7-4890-98a6-35e00445356c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:26.992988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:26.993606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:26.994153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:26.994432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:26.995172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:26.997418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:27.157702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1013/2047] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:26 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:27.161385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:27.174064 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2545b9-9944-4ad6-aacb-e268e80adced admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:27.178406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:27.264024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:27.264943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:27.265325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:27.265667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:27.266425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:27.268616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:27.364047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1035/2048] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:27 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:27.365410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:27.385958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99338b4e-0d06-4782-b860-d2ffd59fadc2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:27.393887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:27.393887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:27.393887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:27.393887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:27.394883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:27.396769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:27.410762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1014/2049] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:27.415432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:27.427775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648785e-9049-4c30-8232-89d4aca75f06 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:27.435812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6a409-3224-4da5-b6ce-37b37e3e5605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:27.436635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6a409-3224-4da5-b6ce-37b37e3e5605 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:27.437096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6a409-3224-4da5-b6ce-37b37e3e5605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:27.437425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6a409-3224-4da5-b6ce-37b37e3e5605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:27.438289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6a409-3224-4da5-b6ce-37b37e3e5605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:27.439972 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6a409-3224-4da5-b6ce-37b37e3e5605 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:27.452423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1036/2050] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:27.456504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:27.470424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aa3f0-f9cd-4f89-a3fe-40b1a5f448cf admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:27.475199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:27.476611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:27.476963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:27.477314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] PATH_INFO: `/v3/projects/8054a20f70e6408ebe06c787887d0b16/users/8f2d8cdf0e0d48f4a89a0e85694c3372/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:27.546553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8054a20f70e6408ebe06c787887d0b16, user_id=8f2d8cdf0e0d48f4a89a0e85694c3372, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:27.546553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:27.610724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1015/2051] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:27 2022] PUT /identity/v3/projects/8054a20f70e6408ebe06c787887d0b16/users/8f2d8cdf0e0d48f4a89a0e85694c3372/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:27.619872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb24ee1-f4cb-4f9c-a3ec-807b739ae035 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:27.619872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb24ee1-f4cb-4f9c-a3ec-807b739ae035 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:27.619872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb24ee1-f4cb-4f9c-a3ec-807b739ae035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:27.651543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-ddb24ee1-f4cb-4f9c-a3ec-807b739ae035 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:27.653720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ddb24ee1-f4cb-4f9c-a3ec-807b739ae035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:27.733538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1037/2052] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:27.743565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:27.745568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:27.790442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:27.803996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017adda5-ce2b-4c08-90cb-51b7c9d286e4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:27.807730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:27.808487 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:27.808855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:27.809213 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:27.818313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:27.823040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:27.829927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1016/2053] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:29.068705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5439b-6a8d-420f-a870-5183c4545af1 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:29.069199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5439b-6a8d-420f-a870-5183c4545af1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:29.069299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5439b-6a8d-420f-a870-5183c4545af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:29.098480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-78d5439b-6a8d-420f-a870-5183c4545af1 None None] MFA Rules not processed for user `8f2d8cdf0e0d48f4a89a0e85694c3372`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:29.099839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-78d5439b-6a8d-420f-a870-5183c4545af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:29.165628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1038/2054] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:29.176640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:29.178038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:29.233367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab1c4e-e5c7-416b-81de-1681e462a968 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:29.236890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:29.237496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:29.237772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:29.238044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:29.246425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:29.247836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:29.256354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1017/2055] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:32.333819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:32.334266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:32.334426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:32.359924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] MFA Rules not processed for user `8f2d8cdf0e0d48f4a89a0e85694c3372`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:32.361186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:32.369753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1039/2056] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:33.098751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:33.111944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:33.129548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e67f66-4020-4922-9c1b-3ac2b500e86f service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:33.135967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:33.136657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:33.136940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:33.137218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] PATH_INFO: `/v3/users/9886343b48e84e3e931ba54a08019a3b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:33.138805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9886343b48e84e3e931ba54a08019a3b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:33.141082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:33.261010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:33.267500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:33.277916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b99e08-2904-4312-b9af-db5579d683c0 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:33.284021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:33.284996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:33.290911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:33.290911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] PATH_INFO: `/v3/users/9b149e74681044959e6eba750d9a26cc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:33.290911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b149e74681044959e6eba750d9a26cc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:33.290911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:33.292828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:33.323705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9886343b48e84e3e931ba54a08019a3b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.324412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9886343b48e84e3e931ba54a08019a3b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.437294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9886343b48e84e3e931ba54a08019a3b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.451961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9886343b48e84e3e931ba54a08019a3b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.458922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-369fce65-26cd-44c3-ae93-7638c00a7a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9886343b48e84e3e931ba54a08019a3b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.465599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1018/2057] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:33 2022] DELETE /identity/v3/users/9886343b48e84e3e931ba54a08019a3b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:33.472754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:33.472754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:33.472754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:33.489597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:33.490836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:33.556286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:33.560299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b149e74681044959e6eba750d9a26cc'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.561585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b149e74681044959e6eba750d9a26cc'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.571545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1019/2058] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:33.582141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:33.592512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:33.641370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e36d2a-0519-4e1c-bf6a-23e0bc1f3fce None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:33.644965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68ebce-6cdb-41b6-8aa1-d4392523fd2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:33.645604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ebce-6cdb-41b6-8aa1-d4392523fd2f service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:33.645881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ebce-6cdb-41b6-8aa1-d4392523fd2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:33.646150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68ebce-6cdb-41b6-8aa1-d4392523fd2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:33.655517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68ebce-6cdb-41b6-8aa1-d4392523fd2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:33.656976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68ebce-6cdb-41b6-8aa1-d4392523fd2f service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:33.666414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b149e74681044959e6eba750d9a26cc'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.669126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1020/2059] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:33.681778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b149e74681044959e6eba750d9a26cc'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.682589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b149e74681044959e6eba750d9a26cc'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:33.702074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1040/2060] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:33 2022] DELETE /identity/v3/users/9b149e74681044959e6eba750d9a26cc => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:33.704021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:33.704311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:33.704622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:33.727932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:33.730861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:33.803246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1021/2061] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:33.813164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:33.814705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:33.875704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8d9ac-4ce0-4616-a274-eb4be617a62c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:33.882417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:33.883089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:33.883367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:33.883742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:33.893480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:33.894956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:33.907260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1041/2062] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:34.228443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:34.230472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:34.250611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972ce8-2b4c-4868-b83a-e02960e205d4 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:34.271178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:34.271178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:34.271178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:34.271178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] PATH_INFO: `/v3/projects/2b087605a91049d596c5f386d84f3376` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:34.271533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b087605a91049d596c5f386d84f3376)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:34.271533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:34.427861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b087605a91049d596c5f386d84f3376', 'request_id': 'req-00b665c1-fda4-4440-92b8-77a704c12c18'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.435254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b087605a91049d596c5f386d84f3376', 'request_id': 'req-00b665c1-fda4-4440-92b8-77a704c12c18'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.440350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:34.443511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:34.454717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b087605a91049d596c5f386d84f3376', 'request_id': 'req-00b665c1-fda4-4440-92b8-77a704c12c18'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.458549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f657b5f-3ee2-4563-b5ab-89657bc399ce service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:34.541974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:34.542636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:34.542923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:34.543203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] PATH_INFO: `/v3/projects/3f76109845c547b8a2253df709c91826` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:34.544452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f76109845c547b8a2253df709c91826)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:34.546413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:34.582204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] The token cache is being invalidate because project 2b087605a91049d596c5f386d84f3376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:34.582502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.583352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.585509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1022/2063] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:34 2022] DELETE /identity/v3/projects/2b087605a91049d596c5f386d84f3376 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:34.589971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:34.592236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:34.606962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b665c1-fda4-4440-92b8-77a704c12c18 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:34.611818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:34.612517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:34.612794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:34.613074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] PATH_INFO: `/v3/users/25f0ee3f17e642db841c247eca04941a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:34.614275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f0ee3f17e642db841c247eca04941a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:34.616860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:34.639112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:34.680705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:34.701500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f76109845c547b8a2253df709c91826', 'request_id': 'req-38743f93-2122-45a7-aa07-b301f790e536'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.707910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f76109845c547b8a2253df709c91826', 'request_id': 'req-38743f93-2122-45a7-aa07-b301f790e536'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.713539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f76109845c547b8a2253df709c91826', 'request_id': 'req-38743f93-2122-45a7-aa07-b301f790e536'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.819174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] The token cache is being invalidate because project 3f76109845c547b8a2253df709c91826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:34.829578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.829578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.829578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1042/2064] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:34 2022] DELETE /identity/v3/projects/3f76109845c547b8a2253df709c91826 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:34.829578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:34.829578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:34.863707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f0ee3f17e642db841c247eca04941a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.864419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f0ee3f17e642db841c247eca04941a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.897435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38743f93-2122-45a7-aa07-b301f790e536 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:34.902398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:34.904137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:34.904137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:34.904137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] PATH_INFO: `/v3/users/dca962d7ae3341feb5a53a45cf38ddff` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:34.910109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca962d7ae3341feb5a53a45cf38ddff)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:34.910109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:34.952626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f0ee3f17e642db841c247eca04941a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.965994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f0ee3f17e642db841c247eca04941a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.976022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f0ee3f17e642db841c247eca04941a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:34.982197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1023/2065] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:34 2022] DELETE /identity/v3/users/25f0ee3f17e642db841c247eca04941a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:35.171668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca962d7ae3341feb5a53a45cf38ddff'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.172445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca962d7ae3341feb5a53a45cf38ddff'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.270332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca962d7ae3341feb5a53a45cf38ddff'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.303729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca962d7ae3341feb5a53a45cf38ddff'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.316472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca962d7ae3341feb5a53a45cf38ddff'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.343471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1043/2066] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:34 2022] DELETE /identity/v3/users/dca962d7ae3341feb5a53a45cf38ddff => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:35.475056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:35.476613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:35.557475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78afa58-defe-4d03-9a67-c5ec805e6498 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:35.617420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:35.618184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:35.618289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:35.618525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] PATH_INFO: `/v3/projects/b9c65645583241288a9808e3a531b3a4` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:35.620011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c65645583241288a9808e3a531b3a4)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:35.621999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:35.825337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:35.829082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c65645583241288a9808e3a531b3a4', 'request_id': 'req-5d41213b-c613-43f2-a96c-328b86067ece'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.838632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c65645583241288a9808e3a531b3a4', 'request_id': 'req-5d41213b-c613-43f2-a96c-328b86067ece'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.845823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c65645583241288a9808e3a531b3a4', 'request_id': 'req-5d41213b-c613-43f2-a96c-328b86067ece'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.944458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:35.947137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:35.949205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] The token cache is being invalidate because project b9c65645583241288a9808e3a531b3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:35.949700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.951164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-5d41213b-c613-43f2-a96c-328b86067ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:35.953003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1024/2067] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:35 2022] DELETE /identity/v3/projects/b9c65645583241288a9808e3a531b3a4 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:35.959777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823dd9d-4602-42a3-86a8-377d2d3b0b8d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:35.960427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823dd9d-4602-42a3-86a8-377d2d3b0b8d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:35.960866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823dd9d-4602-42a3-86a8-377d2d3b0b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:35.972785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996cac-56b0-4d80-8780-b94cfca5ea72 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:35.994289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3823dd9d-4602-42a3-86a8-377d2d3b0b8d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:35.995777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3823dd9d-4602-42a3-86a8-377d2d3b0b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:36.014172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:36.044759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.045463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.045927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.046209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] PATH_INFO: `/v3/projects/ff26d49c401c490d9a42f0329a743faa` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.047426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff26d49c401c490d9a42f0329a743faa)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.049365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.071600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1025/2068] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:36.079734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f96b8-6ffb-4330-8779-a89e1b1bcf4c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.080027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f96b8-6ffb-4330-8779-a89e1b1bcf4c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.080302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f96b8-6ffb-4330-8779-a89e1b1bcf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.099876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-178f96b8-6ffb-4330-8779-a89e1b1bcf4c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:36.109437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-178f96b8-6ffb-4330-8779-a89e1b1bcf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:36.109721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1026/2069] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:36.114892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f96b8-6ffb-4330-8779-a89e1b1bcf4c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.125454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f96b8-6ffb-4330-8779-a89e1b1bcf4c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.130713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.132872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.136648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.136648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.136648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:36.136648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.136648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.149424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1027/2070] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:14:36.153695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.164112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1735f2b-9d67-485c-8a82-a0fa45bf8a17 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.169245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.169881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.170166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.170507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.171191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.173194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.203062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff26d49c401c490d9a42f0329a743faa', 'request_id': 'req-dc5face9-234c-414e-a7f1-0b84589e2e3f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.209107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff26d49c401c490d9a42f0329a743faa', 'request_id': 'req-dc5face9-234c-414e-a7f1-0b84589e2e3f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.215649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff26d49c401c490d9a42f0329a743faa', 'request_id': 'req-dc5face9-234c-414e-a7f1-0b84589e2e3f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.306275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1028/2071] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:36 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:36.306476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] The token cache is being invalidate because project ff26d49c401c490d9a42f0329a743faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:36.306476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.306476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.310285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1044/2072] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:35 2022] DELETE /identity/v3/projects/ff26d49c401c490d9a42f0329a743faa => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:36.310444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.311500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.311626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:36.313074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:36.432317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b2df-ec97-4a7f-83ec-63046b8770fe admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.439516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5face9-234c-414e-a7f1-0b84589e2e3f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.443179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.444367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.444674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.444952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.445638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.447584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.458309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.458309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.458309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.458309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] PATH_INFO: `/v3/users/236238eb772d48b6a75f301981a1a597` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.460095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236238eb772d48b6a75f301981a1a597)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.462091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.550980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1029/2073] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:36 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:36.551608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.562999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09895f8-0e35-42eb-a7da-410b9177c1b4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.567761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.568429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.568770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.569554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.570256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.571850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.583039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1030/2074] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:36.587087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.597071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade4bbe-8e13-400d-9e43-5777276b39aa admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.602734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.603767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.604054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.604325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.606053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.608054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.620359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1031/2075] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:36.624648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.635442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa3477-1e1e-4a4a-a896-31ea4a81c52d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.641608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e774337-8a88-4119-8f86-351e805a4570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.642421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774337-8a88-4119-8f86-351e805a4570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.642699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774337-8a88-4119-8f86-351e805a4570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.642971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774337-8a88-4119-8f86-351e805a4570 admin admin] PATH_INFO: `/v3/projects/e2afd21ce4b54919aee0a8d2dcc7ed07/users/7cdbc7699412470a987f82537fe9ca5b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.659805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e774337-8a88-4119-8f86-351e805a4570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2afd21ce4b54919aee0a8d2dcc7ed07, user_id=7cdbc7699412470a987f82537fe9ca5b, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.662615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e774337-8a88-4119-8f86-351e805a4570 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:36.713167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:36.725847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236238eb772d48b6a75f301981a1a597'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.726493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236238eb772d48b6a75f301981a1a597'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.781365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1032/2076] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:36 2022] PUT /identity/v3/projects/e2afd21ce4b54919aee0a8d2dcc7ed07/users/7cdbc7699412470a987f82537fe9ca5b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:36.794994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.795462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.795837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.835879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] MFA Rules not processed for user `7cdbc7699412470a987f82537fe9ca5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:36.837344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:36.837979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236238eb772d48b6a75f301981a1a597'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.857661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236238eb772d48b6a75f301981a1a597'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.858401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236238eb772d48b6a75f301981a1a597'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:36.877402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1045/2077] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:36 2022] DELETE /identity/v3/users/236238eb772d48b6a75f301981a1a597 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:36.880719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.881949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:36.957255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1033/2078] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:36.963290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:36.964874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:36.985046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0d9df-070a-4209-a92e-6c555c84e133 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:36.989984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:36.991341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:36.991620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:36.992082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] PATH_INFO: `/v3/users/03e4ddeb79274bd1a19534d467e01657` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:36.993565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=03e4ddeb79274bd1a19534d467e01657)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:36.997152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.045166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60953497-a68e-42dc-b6e7-9a54cb454522 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.051217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.051854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.052139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.052730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.068947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.068947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.143766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:37.206655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1034/2079] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:36 2022] POST /identity/v3/projects => generated 373 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:37.207672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:37.245986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03e4ddeb79274bd1a19534d467e01657'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.246772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03e4ddeb79274bd1a19534d467e01657'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.302832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4173f35-8897-4c93-a79c-78dda7c6792d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.307604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.308237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.308540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.308814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.309639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.311457 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.341975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03e4ddeb79274bd1a19534d467e01657'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.355698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03e4ddeb79274bd1a19534d467e01657'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.356808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2844984-4624-4429-a8bf-d86f98a95cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03e4ddeb79274bd1a19534d467e01657'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.369756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1046/2080] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:36 2022] DELETE /identity/v3/users/03e4ddeb79274bd1a19534d467e01657 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:37.377765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.378187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.381852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.404461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:37.405730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.451699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1035/2081] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:37 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:37.453695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:37.456993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.469503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6cfdd-208d-466d-b92e-b81a0e494bd1 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.474116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.474241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1047/2082] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:37.476593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.476702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:37.476787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.476787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] PATH_INFO: `/v3/projects/a3a31663bb7b45879cde0bcb8812ea1d` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.481941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.482024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3a31663bb7b45879cde0bcb8812ea1d)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.482024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.489588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06008fff-cc42-44c1-9a50-79c0bcfafb51 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.494642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.495243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.495577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.495858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.496597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.498287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.510676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1048/2083] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:37.511584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:37.513290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.574516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5932af-7aa4-4157-8c36-04dcbefc6e4e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.578440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.579174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.580874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.580874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.589792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.595737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.605158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1049/2084] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:37.609351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:37.621322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f9af1-7ddc-4e18-b1c1-c123a463f79a service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.628330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.629231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.630880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.631170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] PATH_INFO: `/v3/projects/6a575ebdea67418f9cac386daf58ca62/users/2bed08d2b5c44a4fbb7b8025c83f67a9/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.650367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a575ebdea67418f9cac386daf58ca62, user_id=2bed08d2b5c44a4fbb7b8025c83f67a9, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.653560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.670389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3a31663bb7b45879cde0bcb8812ea1d', 'request_id': 'req-be72d904-0a89-460d-8af7-d91465f6e56e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.678687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3a31663bb7b45879cde0bcb8812ea1d', 'request_id': 'req-be72d904-0a89-460d-8af7-d91465f6e56e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.685341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3a31663bb7b45879cde0bcb8812ea1d', 'request_id': 'req-be72d904-0a89-460d-8af7-d91465f6e56e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.724488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1050/2085] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:37 2022] PUT /identity/v3/projects/6a575ebdea67418f9cac386daf58ca62/users/2bed08d2b5c44a4fbb7b8025c83f67a9/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:37.729344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:37.754947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] The token cache is being invalidate because project a3a31663bb7b45879cde0bcb8812ea1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:37.754947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.754947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-be72d904-0a89-460d-8af7-d91465f6e56e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:37.754947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1036/2086] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:37 2022] DELETE /identity/v3/projects/a3a31663bb7b45879cde0bcb8812ea1d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:37.757904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebb10-959f-4d7e-9631-613f3a73ab57 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.758192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebb10-959f-4d7e-9631-613f3a73ab57 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.758492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebb10-959f-4d7e-9631-613f3a73ab57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.781454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-91eebb10-959f-4d7e-9631-613f3a73ab57 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:37.783296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-91eebb10-959f-4d7e-9631-613f3a73ab57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.798769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc0cd5-f5de-49cf-98fa-86a4093b6ed0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.804502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.805215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.805500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.805777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.806534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.808204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.823440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1051/2087] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:37.824577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:37.828592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.842732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7e3a1-f4fa-4ed1-abd5-fd3a426f0ab1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:37.848667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db95f65-dd99-458e-b55e-5fd652114b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:37.849270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db95f65-dd99-458e-b55e-5fd652114b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.849555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db95f65-dd99-458e-b55e-5fd652114b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.850366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db95f65-dd99-458e-b55e-5fd652114b7c admin admin] PATH_INFO: `/v3/projects/6a575ebdea67418f9cac386daf58ca62/users/2bed08d2b5c44a4fbb7b8025c83f67a9/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.852674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db95f65-dd99-458e-b55e-5fd652114b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a575ebdea67418f9cac386daf58ca62, user_id=2bed08d2b5c44a4fbb7b8025c83f67a9, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:37.861939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db95f65-dd99-458e-b55e-5fd652114b7c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:37.871465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1052/2088] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:37 2022] PUT /identity/v3/projects/6a575ebdea67418f9cac386daf58ca62/users/2bed08d2b5c44a4fbb7b8025c83f67a9/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:37.883657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.884048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.884327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.892520 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1037/2089] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:37.894861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b19e98-f656-4a5a-9966-8e911ef5c1ab None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:37.895320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b19e98-f656-4a5a-9966-8e911ef5c1ab None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:37.895748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b19e98-f656-4a5a-9966-8e911ef5c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:37.921167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b5b19e98-f656-4a5a-9966-8e911ef5c1ab None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:37.922623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b5b19e98-f656-4a5a-9966-8e911ef5c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.923094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] MFA Rules not processed for user `2bed08d2b5c44a4fbb7b8025c83f67a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:37.924422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:37.996228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1038/2090] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:38.000764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b19e98-f656-4a5a-9966-8e911ef5c1ab None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.012589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b19e98-f656-4a5a-9966-8e911ef5c1ab None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.013057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1053/2091] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:38.017385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.018375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.018665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.018984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.019701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:38.020002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.022463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.022625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.024357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:38.035669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1039/2092] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:14:38.046916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.057546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42efbd4-93f0-473b-84ed-5b8803cad13a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.063591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.064349 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.064669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.064943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.065873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.068284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.093888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3585-d944-4a91-8d8e-9a7a3e0ff927 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.098181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.098885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.100701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.100701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.102722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:38.165697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1040/2093] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:38 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:38.166966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.168944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:38.181889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.184419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.201994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1054/2094] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:38.202930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.219158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffa055-23d0-4c50-bb39-ca5934de6f85 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.253416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579954d-303e-4f6d-b55f-1342aa86c23c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.266227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.267477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.267767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.268049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] PATH_INFO: `/v3/projects/785f72fbecb042cab2a97bc5193c15d3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.273595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=785f72fbecb042cab2a97bc5193c15d3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.275849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.348341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.349137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.349465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.349682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.350434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.353696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.419491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785f72fbecb042cab2a97bc5193c15d3', 'request_id': 'req-fc8a534f-7095-411e-8957-fba412f262e2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.425900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785f72fbecb042cab2a97bc5193c15d3', 'request_id': 'req-fc8a534f-7095-411e-8957-fba412f262e2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.432418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785f72fbecb042cab2a97bc5193c15d3', 'request_id': 'req-fc8a534f-7095-411e-8957-fba412f262e2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.489916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1055/2095] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:38 2022] POST /identity/v3/users => generated 296 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:38.495163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.516444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011de6-a454-484f-8de8-f3a34dc4b4b3 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.516629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:38.523355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] The token cache is being invalidate because project 785f72fbecb042cab2a97bc5193c15d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:38.523735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.524441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.529125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1041/2096] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:38 2022] DELETE /identity/v3/projects/785f72fbecb042cab2a97bc5193c15d3 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:38.530131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.531742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:38.602852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.603614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.603986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.604369 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.605182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.606865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.609290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a534f-7095-411e-8957-fba412f262e2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.614387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.615039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.615328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.616224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] PATH_INFO: `/v3/users/6b4bcdb325794170869bbfe6ad5c97af` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.617474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4bcdb325794170869bbfe6ad5c97af)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.619735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.622615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1056/2097] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:38.623539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.625413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:38.642884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e4b7e-bfa8-432a-b863-c272fa7c2d66 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.643876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.644531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.644817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.645095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.645802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.647949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.658723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1057/2098] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:38.663512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:38.673431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7c3e4-8c8f-41a3-b4f4-519ff6e6f791 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:38.678543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c10d1-3079-4192-96a4-9a48d42f8641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:38.679267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c10d1-3079-4192-96a4-9a48d42f8641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.679604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c10d1-3079-4192-96a4-9a48d42f8641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.679937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c10d1-3079-4192-96a4-9a48d42f8641 admin admin] PATH_INFO: `/v3/projects/db7b2778ff8948f6826d45020132363f/users/2f359aa5a3264eafa8eeab4f1701f59b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.695692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c10d1-3079-4192-96a4-9a48d42f8641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7b2778ff8948f6826d45020132363f, user_id=2f359aa5a3264eafa8eeab4f1701f59b, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:38.698365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c10d1-3079-4192-96a4-9a48d42f8641 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:38.778493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1058/2099] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:38 2022] PUT /identity/v3/projects/db7b2778ff8948f6826d45020132363f/users/2f359aa5a3264eafa8eeab4f1701f59b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:38.784658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:38.785005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:38.785580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:38.805458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4bcdb325794170869bbfe6ad5c97af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.806313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4bcdb325794170869bbfe6ad5c97af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.823460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] MFA Rules not processed for user `2f359aa5a3264eafa8eeab4f1701f59b`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:38.825992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:38.826770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:38.900075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1059/2100] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:38.911208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4bcdb325794170869bbfe6ad5c97af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.924612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4bcdb325794170869bbfe6ad5c97af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.932055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4bcdb325794170869bbfe6ad5c97af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:38.938963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1042/2101] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:38 2022] DELETE /identity/v3/users/6b4bcdb325794170869bbfe6ad5c97af => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:39.116027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:39.117508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:39.202217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9fa30-3a3b-49ab-b158-a011b14a8c85 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:39.263989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26315cd-f6cb-4754-bf46-6392b92bba37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:39.264727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26315cd-f6cb-4754-bf46-6392b92bba37 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:39.265078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26315cd-f6cb-4754-bf46-6392b92bba37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:39.265485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26315cd-f6cb-4754-bf46-6392b92bba37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:39.267269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e26315cd-f6cb-4754-bf46-6392b92bba37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:39.327252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26315cd-f6cb-4754-bf46-6392b92bba37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:39.328686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26315cd-f6cb-4754-bf46-6392b92bba37 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:39.340986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1060/2102] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:39.412857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:39.414310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:39.488967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d1c0c-e212-4933-86c8-baf6547a8cdd admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:39.494438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:39.495087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:39.495370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:39.495664 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] PATH_INFO: `/v3/projects/4ca56bfd0e364aa8857e677f5d83fbb6` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:39.496850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca56bfd0e364aa8857e677f5d83fbb6)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:39.498811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:39.614706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:39.633337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca56bfd0e364aa8857e677f5d83fbb6', 'request_id': 'req-727d06c1-ca1d-40f9-8833-326ffd432366'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:39.639551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca56bfd0e364aa8857e677f5d83fbb6', 'request_id': 'req-727d06c1-ca1d-40f9-8833-326ffd432366'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:39.645142 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca56bfd0e364aa8857e677f5d83fbb6', 'request_id': 'req-727d06c1-ca1d-40f9-8833-326ffd432366'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:39.814523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] The token cache is being invalidate because project 4ca56bfd0e364aa8857e677f5d83fbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:39.815130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:39.815538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-727d06c1-ca1d-40f9-8833-326ffd432366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:39.817078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1043/2103] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:39 2022] DELETE /identity/v3/projects/4ca56bfd0e364aa8857e677f5d83fbb6 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:39.823019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d3b1-187a-4b6c-abf7-9d1d516bc65e None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:39.823312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d3b1-187a-4b6c-abf7-9d1d516bc65e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:39.823588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d3b1-187a-4b6c-abf7-9d1d516bc65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:39.843050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4a94d3b1-187a-4b6c-abf7-9d1d516bc65e None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:39.844247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4a94d3b1-187a-4b6c-abf7-9d1d516bc65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:39.903891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1061/2104] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:39.911468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4b39e-2ee5-48a4-bba8-466bf766ef17 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:39.911761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4b39e-2ee5-48a4-bba8-466bf766ef17 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:39.912043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4b39e-2ee5-48a4-bba8-466bf766ef17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:39.932540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-10b4b39e-2ee5-48a4-bba8-466bf766ef17 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:39.933754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-10b4b39e-2ee5-48a4-bba8-466bf766ef17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:39.941667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1044/2105] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:39.945893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d3b1-187a-4b6c-abf7-9d1d516bc65e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:39.956005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d3b1-187a-4b6c-abf7-9d1d516bc65e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:39.961324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:39.961936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:39.962219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:39.962619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:39.963322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:39.963629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:39.965277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:39.977195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1062/2106] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:14:39.981758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4b39e-2ee5-48a4-bba8-466bf766ef17 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:39.991596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4b39e-2ee5-48a4-bba8-466bf766ef17 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:39.996530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:39.997137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:39.997419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:39.997695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:39.998364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:40.000261 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:40.111108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1045/2107] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:39 2022] POST /identity/v3/projects => generated 399 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:40.115364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:40.126475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74944b03-83dc-4206-9e2f-487a61ec6f80 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:40.214191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:40.214191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:40.214191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:40.214545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:40.215327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:40.217584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:40.300934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1063/2108] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:40 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:40.304959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:40.315245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee510d8e-c8e4-46d8-a6d8-a399aeb57c8d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:40.319778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:40.320397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:40.320680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:40.320955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:40.321653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:40.323239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:40.333084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1046/2109] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:40.337755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:40.347937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81225e8-5768-43e7-a9ff-473979bd2e81 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:40.353312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89878fa-c22f-43d8-a73c-b7c41d97343b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:40.353913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89878fa-c22f-43d8-a73c-b7c41d97343b admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:40.354194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89878fa-c22f-43d8-a73c-b7c41d97343b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:40.354486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89878fa-c22f-43d8-a73c-b7c41d97343b admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:40.355161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89878fa-c22f-43d8-a73c-b7c41d97343b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:40.356839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89878fa-c22f-43d8-a73c-b7c41d97343b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:40.367286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1064/2110] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:40.371697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:40.381605 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2eeff-d0a3-45ad-a8ac-caa93c7a9399 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:40.386152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:40.386800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:40.387083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:40.387359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] PATH_INFO: `/v3/projects/1bac6a1a7b9a4c09bb3069963aae531c/users/e8d1ef72a14d4e608739919c3d96579c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:40.402432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bac6a1a7b9a4c09bb3069963aae531c, user_id=e8d1ef72a14d4e608739919c3d96579c, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:40.407416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:40.529924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1047/2111] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:40 2022] PUT /identity/v3/projects/1bac6a1a7b9a4c09bb3069963aae531c/users/e8d1ef72a14d4e608739919c3d96579c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:40.540340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5ef4a-4589-4c14-9caa-c0aae971ddd9 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:40.540655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5ef4a-4589-4c14-9caa-c0aae971ddd9 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:40.540930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5ef4a-4589-4c14-9caa-c0aae971ddd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:40.567633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-53f5ef4a-4589-4c14-9caa-c0aae971ddd9 None None] MFA Rules not processed for user `e8d1ef72a14d4e608739919c3d96579c`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:40.569400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-53f5ef4a-4589-4c14-9caa-c0aae971ddd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:40.587657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:40.634762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1065/2112] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:40.639227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:40.706001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e8323-5e13-4337-8ba6-c1fcc8b5921d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:40.710656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:40.711265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:40.711546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:40.711818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:40.712517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:40.714398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:40.810586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1048/2113] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:40 2022] POST /identity/v3/projects => generated 399 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:40.814438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5ef4a-4589-4c14-9caa-c0aae971ddd9 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:40.825692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5ef4a-4589-4c14-9caa-c0aae971ddd9 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:40.885059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:40.885685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:40.885966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:40.886237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:40.886912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:40.889023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:40.959017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:40.992010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1066/2114] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:40 2022] POST /identity/v3/users => generated 306 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:40.996122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:41.007081 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874404af-d389-4fe9-921e-ec39928e69f8 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:41.012100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:41.012753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:41.013037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:41.013312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:41.014010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:41.015595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:41.026645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1049/2115] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:41.030878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:41.033209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:41.043021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cf2b0-5a0e-469f-91e4-06dc8b786fb0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:41.047550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:41.048217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:41.048530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:41.048804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] PATH_INFO: `/v3/projects/b247138df74e4078a4275c4c3e4210d9/users/34c442d0c3b04464b13dbf6b1406ed6c/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:41.064496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b247138df74e4078a4275c4c3e4210d9, user_id=34c442d0c3b04464b13dbf6b1406ed6c, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:41.067145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:41.174932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1067/2116] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:41 2022] PUT /identity/v3/projects/b247138df74e4078a4275c4c3e4210d9/users/34c442d0c3b04464b13dbf6b1406ed6c/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:41.179102 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:41.189149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad5cc4-3dfc-4942-a76a-8293dcdda186 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:41.256621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a096d-6feb-4939-9c36-78a5be60451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:41.262633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a096d-6feb-4939-9c36-78a5be60451b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:41.262633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a096d-6feb-4939-9c36-78a5be60451b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:41.262633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a096d-6feb-4939-9c36-78a5be60451b admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:41.262633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a096d-6feb-4939-9c36-78a5be60451b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:41.262633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a096d-6feb-4939-9c36-78a5be60451b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:41.272275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1050/2117] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:41.276150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:41.291389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af700c74-1780-4404-a0bd-01692261cf44 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:41.296112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:41.296808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:41.297117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:41.297364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] PATH_INFO: `/v3/projects/b247138df74e4078a4275c4c3e4210d9/users/34c442d0c3b04464b13dbf6b1406ed6c/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:41.299373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b247138df74e4078a4275c4c3e4210d9, user_id=34c442d0c3b04464b13dbf6b1406ed6c, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:41.302843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:41.313567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1068/2118] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:41 2022] PUT /identity/v3/projects/b247138df74e4078a4275c4c3e4210d9/users/34c442d0c3b04464b13dbf6b1406ed6c/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:41.340706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4248323-aba3-4d1c-82a8-c62ef7fd3f4a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:41.341087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4248323-aba3-4d1c-82a8-c62ef7fd3f4a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:41.341432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4248323-aba3-4d1c-82a8-c62ef7fd3f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:41.369150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a4248323-aba3-4d1c-82a8-c62ef7fd3f4a None None] MFA Rules not processed for user `34c442d0c3b04464b13dbf6b1406ed6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:41.370418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a4248323-aba3-4d1c-82a8-c62ef7fd3f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:41.438344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1051/2119] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:41 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:41.449617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:41.450833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:41.452581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:41.478116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:41.511438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29249d1-c080-43dc-8c87-530f475586b1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:41.515126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:41.515754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:41.516070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:41.516441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:41.526120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:41.527600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:41.537191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1069/2120] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:41.563672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:41.807826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4248323-aba3-4d1c-82a8-c62ef7fd3f4a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:41.815123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:41.819516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4248323-aba3-4d1c-82a8-c62ef7fd3f4a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:41.823164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc71b9b-1d75-4878-be6a-83b602df135b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:41.823838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc71b9b-1d75-4878-be6a-83b602df135b service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:41.824117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc71b9b-1d75-4878-be6a-83b602df135b service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:41.824441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc71b9b-1d75-4878-be6a-83b602df135b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:41.842801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:41.879766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc71b9b-1d75-4878-be6a-83b602df135b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:41.881836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc71b9b-1d75-4878-be6a-83b602df135b service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:41.890650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1052/2121] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:48.472111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:48.473785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:48.548501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996ab1-9b0c-469b-9553-06347489ed2b service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:48.554531 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:48.555195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:48.555458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:48.555890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] PATH_INFO: `/v3/users/7cdbc7699412470a987f82537fe9ca5b` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:48.557107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdbc7699412470a987f82537fe9ca5b)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:48.559024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:48.764913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdbc7699412470a987f82537fe9ca5b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:48.765670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdbc7699412470a987f82537fe9ca5b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:48.836280 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdbc7699412470a987f82537fe9ca5b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:48.848951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdbc7699412470a987f82537fe9ca5b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:48.849664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdbc7699412470a987f82537fe9ca5b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:48.862536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1070/2122] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:48 2022] DELETE /identity/v3/users/7cdbc7699412470a987f82537fe9ca5b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:48.867721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:48.868242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:48.868242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:48.887176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:48.888355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:48.948594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1053/2123] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:48.958281 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:48.959727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:49.019952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9109c23-774c-404f-8fc2-b47e9aece71f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:49.023977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:49.024631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:49.024921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:49.025214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:49.032685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:49.035012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:49.036505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:49.050779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1071/2124] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:49.339567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:49.341695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:49.365248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32898ef-d68b-4b83-a321-b3058d2656a7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:49.385347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:49.386018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:49.386303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:49.386579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] PATH_INFO: `/v3/users/e8d1ef72a14d4e608739919c3d96579c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:49.387795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d1ef72a14d4e608739919c3d96579c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:49.389758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:49.538100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:49.539971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:49.553219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c60ea-c167-45b1-b97b-5e5175ed563e service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:49.557989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:49.558634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:49.558915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:49.559189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] PATH_INFO: `/v3/projects/e2afd21ce4b54919aee0a8d2dcc7ed07` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:49.560418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2afd21ce4b54919aee0a8d2dcc7ed07)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:49.562387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:49.565194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:49.579615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d1ef72a14d4e608739919c3d96579c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.580604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d1ef72a14d4e608739919c3d96579c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.661133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d1ef72a14d4e608739919c3d96579c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.674290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d1ef72a14d4e608739919c3d96579c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.681843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-484e7363-18c2-423e-9530-fd8b56f01538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d1ef72a14d4e608739919c3d96579c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.690285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1054/2125] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:49 2022] DELETE /identity/v3/users/e8d1ef72a14d4e608739919c3d96579c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:49.694988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:49.695274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:49.695543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:49.720731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:49.722125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:49.730768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2afd21ce4b54919aee0a8d2dcc7ed07', 'request_id': 'req-842e6b79-25c0-4e94-8286-fcf108782654'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.738198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2afd21ce4b54919aee0a8d2dcc7ed07', 'request_id': 'req-842e6b79-25c0-4e94-8286-fcf108782654'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.745379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2afd21ce4b54919aee0a8d2dcc7ed07', 'request_id': 'req-842e6b79-25c0-4e94-8286-fcf108782654'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.805101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1055/2126] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:49.813120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:49.814520 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:49.833126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] The token cache is being invalidate because project e2afd21ce4b54919aee0a8d2dcc7ed07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:49.833422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.834059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:49.837883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1072/2127] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:49 2022] DELETE /identity/v3/projects/e2afd21ce4b54919aee0a8d2dcc7ed07 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:49.840122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:49.841585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:49.854440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e6b79-25c0-4e94-8286-fcf108782654 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:49.859277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:49.859939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:49.860344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:49.860664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] PATH_INFO: `/v3/users/2bed08d2b5c44a4fbb7b8025c83f67a9` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:49.861861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bed08d2b5c44a4fbb7b8025c83f67a9)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:49.863781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:49.876316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2537a2-3130-497f-819f-a96050b39432 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:49.880050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:49.880698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:49.881089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:49.881371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:49.883791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:49.895815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:49.897827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:49.907448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1056/2128] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:50.072494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bed08d2b5c44a4fbb7b8025c83f67a9'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.073135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bed08d2b5c44a4fbb7b8025c83f67a9'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.144614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bed08d2b5c44a4fbb7b8025c83f67a9'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.158111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bed08d2b5c44a4fbb7b8025c83f67a9'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.158863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bed08d2b5c44a4fbb7b8025c83f67a9'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.172560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1073/2129] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:49 2022] DELETE /identity/v3/users/2bed08d2b5c44a4fbb7b8025c83f67a9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:50.325049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:50.326618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:50.346048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144f56-e9b5-4c51-8d74-1c5a3a4e2d0b service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:50.431430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:50.438370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:50.439109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:50.439455 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:50.439795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] PATH_INFO: `/v3/projects/1bac6a1a7b9a4c09bb3069963aae531c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:50.441192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bac6a1a7b9a4c09bb3069963aae531c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:50.443290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:50.565553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bac6a1a7b9a4c09bb3069963aae531c', 'request_id': 'req-4fc84e29-006d-4f07-9f29-3850344e4ff8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.572915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bac6a1a7b9a4c09bb3069963aae531c', 'request_id': 'req-4fc84e29-006d-4f07-9f29-3850344e4ff8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.578367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bac6a1a7b9a4c09bb3069963aae531c', 'request_id': 'req-4fc84e29-006d-4f07-9f29-3850344e4ff8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.643314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] The token cache is being invalidate because project 1bac6a1a7b9a4c09bb3069963aae531c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:50.643615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.644255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.645910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1057/2130] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:50 2022] DELETE /identity/v3/projects/1bac6a1a7b9a4c09bb3069963aae531c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:50.649935 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:50.651406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:50.667343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea854e7-5011-4c3e-be0e-b3fc3b0a3aca admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:50.698145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:50.699942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:50.734425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:50.751591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:50.752566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:50.752850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:50.753128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] PATH_INFO: `/v3/users/34c442d0c3b04464b13dbf6b1406ed6c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:50.754928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c442d0c3b04464b13dbf6b1406ed6c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:50.759012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:50.783720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc84e29-006d-4f07-9f29-3850344e4ff8 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:50.788780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:50.789461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:50.789743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:50.790029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] PATH_INFO: `/v3/projects/6a575ebdea67418f9cac386daf58ca62` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:50.791625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a575ebdea67418f9cac386daf58ca62)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:50.793522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:50.969272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a575ebdea67418f9cac386daf58ca62', 'request_id': 'req-d5ebe30a-a148-4072-a90e-68b3f71598a2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.975940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a575ebdea67418f9cac386daf58ca62', 'request_id': 'req-d5ebe30a-a148-4072-a90e-68b3f71598a2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.980937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c442d0c3b04464b13dbf6b1406ed6c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.981647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c442d0c3b04464b13dbf6b1406ed6c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:50.981989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a575ebdea67418f9cac386daf58ca62', 'request_id': 'req-d5ebe30a-a148-4072-a90e-68b3f71598a2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.055054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c442d0c3b04464b13dbf6b1406ed6c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.068297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c442d0c3b04464b13dbf6b1406ed6c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.069038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f0009b45-dc9c-4788-8eed-bb02a27e6fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c442d0c3b04464b13dbf6b1406ed6c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.081555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1074/2131] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:50 2022] DELETE /identity/v3/users/34c442d0c3b04464b13dbf6b1406ed6c => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:51.112406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] The token cache is being invalidate because project 6a575ebdea67418f9cac386daf58ca62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:51.112712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.113372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d5ebe30a-a148-4072-a90e-68b3f71598a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.115441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1058/2132] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:50 2022] DELETE /identity/v3/projects/6a575ebdea67418f9cac386daf58ca62 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:51.121305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73bd2a-dab8-43ea-a272-85537179bebb None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.136477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73bd2a-dab8-43ea-a272-85537179bebb None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.136477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73bd2a-dab8-43ea-a272-85537179bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.143491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-eb73bd2a-dab8-43ea-a272-85537179bebb None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:51.144855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-eb73bd2a-dab8-43ea-a272-85537179bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:51.207741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1075/2133] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:51.215502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd0d54-8abf-4786-a37b-414e5e23c146 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.215792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd0d54-8abf-4786-a37b-414e5e23c146 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.216073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd0d54-8abf-4786-a37b-414e5e23c146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.236334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6cdd0d54-8abf-4786-a37b-414e5e23c146 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:51.237572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6cdd0d54-8abf-4786-a37b-414e5e23c146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:51.246445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1059/2134] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:51.250747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73bd2a-dab8-43ea-a272-85537179bebb None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.260554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73bd2a-dab8-43ea-a272-85537179bebb None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.265661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.266289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.266573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.266852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.267554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:51.267862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.269509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.281713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1076/2135] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:14:51.286003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd0d54-8abf-4786-a37b-414e5e23c146 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.296548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd0d54-8abf-4786-a37b-414e5e23c146 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.301815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.302436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.302719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.302989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.304403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.306350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.312110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:51.389668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1060/2136] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:51.391151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.403750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2391f1-3cbb-4907-96bd-0c93b2dd06d3 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.457388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:51.463987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.464651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.464937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.465214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.465892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.467917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.490933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.492772 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:51.506260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ad5c5-1aec-4ec6-832b-28c60c81ff2a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.510974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.511594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.512798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.512798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] PATH_INFO: `/v3/projects/b247138df74e4078a4275c4c3e4210d9` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.513316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b247138df74e4078a4275c4c3e4210d9)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.515244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.543804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1077/2137] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:51.545652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.555446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb63d5-cdff-4cff-a0f8-b061a33425cf admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.560121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.560756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.561039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.561316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.562003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.563580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.573951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1078/2138] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:51.577957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.587805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426685d1-cb19-4f61-9863-8f41cb7e1a44 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.592369 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.593002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.593229 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.593505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.594169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.595688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.606597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1079/2139] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:51.610493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.619974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b4b7e-bed9-4901-b99c-64d00c4e0b6e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.624597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6230305-4c7c-4414-a150-bb8c02582324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.625400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230305-4c7c-4414-a150-bb8c02582324 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.625701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230305-4c7c-4414-a150-bb8c02582324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.625978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6230305-4c7c-4414-a150-bb8c02582324 admin admin] PATH_INFO: `/v3/projects/0b9c0064463b49fdb26e5c6dbcee79fb/users/ce65af9e391841bb8f077ef49a115115/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.643694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6230305-4c7c-4414-a150-bb8c02582324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9c0064463b49fdb26e5c6dbcee79fb, user_id=ce65af9e391841bb8f077ef49a115115, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.646422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6230305-4c7c-4414-a150-bb8c02582324 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.648672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b247138df74e4078a4275c4c3e4210d9', 'request_id': 'req-a96e4186-4fab-4634-b44b-c01c49339a1a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.659411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b247138df74e4078a4275c4c3e4210d9', 'request_id': 'req-a96e4186-4fab-4634-b44b-c01c49339a1a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.667387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b247138df74e4078a4275c4c3e4210d9', 'request_id': 'req-a96e4186-4fab-4634-b44b-c01c49339a1a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.713312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1080/2140] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:51 2022] PUT /identity/v3/projects/0b9c0064463b49fdb26e5c6dbcee79fb/users/ce65af9e391841bb8f077ef49a115115/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:51.724503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.724794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.725050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.732119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] The token cache is being invalidate because project b247138df74e4078a4275c4c3e4210d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:51.732697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.733831 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-a96e4186-4fab-4634-b44b-c01c49339a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:51.737839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1061/2141] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:51 2022] DELETE /identity/v3/projects/b247138df74e4078a4275c4c3e4210d9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:51.748059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d05736-b5b0-41be-bc27-7d5013603374 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.748576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d05736-b5b0-41be-bc27-7d5013603374 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.749181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d05736-b5b0-41be-bc27-7d5013603374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.762530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] MFA Rules not processed for user `ce65af9e391841bb8f077ef49a115115`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:51.763899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:51.797005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-12d05736-b5b0-41be-bc27-7d5013603374 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:51.798935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-12d05736-b5b0-41be-bc27-7d5013603374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:51.852593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1081/2142] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:51.860410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.863950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:51.888812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1062/2143] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:51.896526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b938d-aedc-4695-8aa9-7a4ce470d574 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.896812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b938d-aedc-4695-8aa9-7a4ce470d574 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.897109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b938d-aedc-4695-8aa9-7a4ce470d574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.918882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3c8b938d-aedc-4695-8aa9-7a4ce470d574 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:51.920106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3c8b938d-aedc-4695-8aa9-7a4ce470d574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:51.928362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1063/2144] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:51.932467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b938d-aedc-4695-8aa9-7a4ce470d574 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.942175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b938d-aedc-4695-8aa9-7a4ce470d574 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.946397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e71fc-0482-492d-b62a-63bc0b04026d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:51.947788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.950105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.950105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.950105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.950105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:51.950105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.950439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:51.951134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:51.951431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:51.951597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.951746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:51.970090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:51.970287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1064/2145] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:14:51.970707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:51.974513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:51.983061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1082/2146] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:51.993180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d523f-6093-421b-8897-915ab3db4011 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:52.001692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:52.002848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.003313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.003788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.005277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:52.009746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:52.108365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1065/2147] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:51 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:52.111635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:52.123989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3747bd2-3382-4f0c-bf2a-04044bb61a4f service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:52.167539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:52.195482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:52.196260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.196498 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.196777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.197452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:52.199393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:52.260487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1083/2148] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:52 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:52.264467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:52.275469 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6665ed5-f1d7-4b04-bb7a-d7e2640ce695 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:52.280171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:52.280893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.281178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.281455 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.282156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:52.283746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:52.294868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1066/2149] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:52.298816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:52.309413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403f237-816e-4fd7-b42f-a0fbe9229177 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:52.314353 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aa9bf-7093-4951-bdb9-1ba003380918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:52.314971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aa9bf-7093-4951-bdb9-1ba003380918 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.315294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aa9bf-7093-4951-bdb9-1ba003380918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.315534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aa9bf-7093-4951-bdb9-1ba003380918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.316235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aa9bf-7093-4951-bdb9-1ba003380918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:52.317994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aa9bf-7093-4951-bdb9-1ba003380918 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:52.328296 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1084/2150] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:52.332431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:52.343311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff4d13-e235-4b40-a53a-526034bd0764 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:52.348101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:52.348944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.349222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.349500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] PATH_INFO: `/v3/projects/2377ba6e7f2c421caea03e2e82cee73c/users/efec8bbdf0cb481b84471f11d6a0a6b6/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.365408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2377ba6e7f2c421caea03e2e82cee73c, user_id=efec8bbdf0cb481b84471f11d6a0a6b6, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:52.368045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:52.370528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:52.484200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1067/2151] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:52 2022] PUT /identity/v3/projects/2377ba6e7f2c421caea03e2e82cee73c/users/efec8bbdf0cb481b84471f11d6a0a6b6/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:52.495204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676dd31-8d4b-4290-b562-a928a1237966 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.495331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676dd31-8d4b-4290-b562-a928a1237966 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.495550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676dd31-8d4b-4290-b562-a928a1237966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.530411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-9676dd31-8d4b-4290-b562-a928a1237966 None None] MFA Rules not processed for user `efec8bbdf0cb481b84471f11d6a0a6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:52.536249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9676dd31-8d4b-4290-b562-a928a1237966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:52.603380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1085/2152] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:52.641593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:52.700545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606f89-1724-4486-9b65-44b9d874e6c0 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:52.704765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f796591-5d7e-4a67-b9c5-5194a8840328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:52.705533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f796591-5d7e-4a67-b9c5-5194a8840328 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.705859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f796591-5d7e-4a67-b9c5-5194a8840328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.706124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f796591-5d7e-4a67-b9c5-5194a8840328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.717745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f796591-5d7e-4a67-b9c5-5194a8840328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:52.717745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f796591-5d7e-4a67-b9c5-5194a8840328 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:52.726002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1068/2153] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:52.750391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676dd31-8d4b-4290-b562-a928a1237966 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:52.822904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676dd31-8d4b-4290-b562-a928a1237966 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:52.827762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:52.828420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:52.828705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:52.828980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] PATH_INFO: `/v3/users/efec8bbdf0cb481b84471f11d6a0a6b6` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:52.830216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efec8bbdf0cb481b84471f11d6a0a6b6)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:52.834379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:53.008974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:53.011201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efec8bbdf0cb481b84471f11d6a0a6b6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:53.011836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efec8bbdf0cb481b84471f11d6a0a6b6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:53.054575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efec8bbdf0cb481b84471f11d6a0a6b6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:53.069397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efec8bbdf0cb481b84471f11d6a0a6b6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:53.070114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efec8bbdf0cb481b84471f11d6a0a6b6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:53.082435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1086/2154] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:52 2022] DELETE /identity/v3/users/efec8bbdf0cb481b84471f11d6a0a6b6 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:53.087620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:53.088189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:53.088189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:53.107850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:53.109189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:53.169466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1069/2155] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:53.182768 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:53.184605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:53.277305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3da6a-7845-4443-b82f-ec9905fb2ab5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:53.281403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:53.282231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:53.282598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:53.282938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:53.294269 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:53.295716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:53.306488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1087/2156] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:53.862292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:53.863811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:53.879908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c9966-90fe-44eb-9453-574fc8a1cd56 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:53.893573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:53.894233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:53.894528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:53.894814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] PATH_INFO: `/v3/projects/2377ba6e7f2c421caea03e2e82cee73c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:53.896655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2377ba6e7f2c421caea03e2e82cee73c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:53.906431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.050035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2377ba6e7f2c421caea03e2e82cee73c', 'request_id': 'req-0ed5d128-b515-4485-8ed9-9242d2786e9f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.056243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2377ba6e7f2c421caea03e2e82cee73c', 'request_id': 'req-0ed5d128-b515-4485-8ed9-9242d2786e9f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.062167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2377ba6e7f2c421caea03e2e82cee73c', 'request_id': 'req-0ed5d128-b515-4485-8ed9-9242d2786e9f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.087439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.089395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:54.103999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28842c-4d2f-4531-9e06-1570ec62dfa4 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.164184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.164983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.165266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.165560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] PATH_INFO: `/v3/users/3f445dd5360444639684f0f7a213e2d6` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.166725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f445dd5360444639684f0f7a213e2d6)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.168889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.179031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] The token cache is being invalidate because project 2377ba6e7f2c421caea03e2e82cee73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:54.179919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.181941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0ed5d128-b515-4485-8ed9-9242d2786e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.183515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1070/2157] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:53 2022] DELETE /identity/v3/projects/2377ba6e7f2c421caea03e2e82cee73c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:54.188621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd3b17-1004-42ce-a6df-579ab3903d21 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.189069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd3b17-1004-42ce-a6df-579ab3903d21 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.189499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd3b17-1004-42ce-a6df-579ab3903d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.223116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-42bd3b17-1004-42ce-a6df-579ab3903d21 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:54.224236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-42bd3b17-1004-42ce-a6df-579ab3903d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:54.232958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1071/2158] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:54.242135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812a5bb-7086-40f6-a3fc-c7bec77624f7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.242420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812a5bb-7086-40f6-a3fc-c7bec77624f7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.242690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812a5bb-7086-40f6-a3fc-c7bec77624f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.263685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b812a5bb-7086-40f6-a3fc-c7bec77624f7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:54.265270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b812a5bb-7086-40f6-a3fc-c7bec77624f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:54.274689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1072/2159] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:54.278656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812a5bb-7086-40f6-a3fc-c7bec77624f7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.291413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812a5bb-7086-40f6-a3fc-c7bec77624f7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.297361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.298173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.298559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.298991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.299834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:54.300274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.305993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.305993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:54.318031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1073/2160] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:14:54.322382 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.334003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342b788-5166-4e84-8ba1-d1ae68033120 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.340640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.341396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.341755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.342221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.343054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.346655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.400351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f445dd5360444639684f0f7a213e2d6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.401557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f445dd5360444639684f0f7a213e2d6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.478122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1074/2161] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:54 2022] POST /identity/v3/projects => generated 357 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:54.481219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.500900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f445dd5360444639684f0f7a213e2d6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.517987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:54.525443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f445dd5360444639684f0f7a213e2d6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.526461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-705a8c14-d293-4882-bc52-fe89891c6845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f445dd5360444639684f0f7a213e2d6'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:54.569783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1088/2162] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:54 2022] DELETE /identity/v3/users/3f445dd5360444639684f0f7a213e2d6 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:54.574303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.574718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.575121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.599049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e1be6-a2af-425b-899e-af5e7471ba89 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.617769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:54.620344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:54.663775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.664442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.664730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.665004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.665668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.667603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.684798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1089/2163] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:54.694444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.696183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:54.754807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fd48d-ed67-4ad2-b2ec-7286d7401983 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.761396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1075/2164] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:54 2022] POST /identity/v3/users => generated 279 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:54.761573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.761573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.761573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.761573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.762695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.764608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:54.773590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.775012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.776869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749707b8-b349-46b0-8fc1-5f6c638ed2bf admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.781794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.782414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.782699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.783116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.783823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.784989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1090/2165] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:54.788957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.799366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1076/2166] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:54.803349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.812896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a546f-ef99-4942-ace2-217d6523cb78 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.817688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f3ee-e021-423d-a95d-57f5b88b036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.818297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f3ee-e021-423d-a95d-57f5b88b036c admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.818579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f3ee-e021-423d-a95d-57f5b88b036c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.819504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f3ee-e021-423d-a95d-57f5b88b036c admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.820049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f3ee-e021-423d-a95d-57f5b88b036c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.821691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f3ee-e021-423d-a95d-57f5b88b036c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.831664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1091/2167] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:54.836595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:54.847865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21081d54-6a29-4ce7-b4f2-e0f8b183d9ce admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:54.852768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:54.853703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.853821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.854028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] PATH_INFO: `/v3/projects/6eb915e83f9c42889c5a1359a42cc2c6/users/e600cec7d1694298bd62db0de116adb1/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.869514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eb915e83f9c42889c5a1359a42cc2c6, user_id=e600cec7d1694298bd62db0de116adb1, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:54.872161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:54.934077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1077/2168] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:54 2022] PUT /identity/v3/projects/6eb915e83f9c42889c5a1359a42cc2c6/users/e600cec7d1694298bd62db0de116adb1/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:54.944916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fbee3-e14f-47e9-91ef-3a4299f00fa6 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:54.945073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fbee3-e14f-47e9-91ef-3a4299f00fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:54.945441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fbee3-e14f-47e9-91ef-3a4299f00fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:54.977760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6c7fbee3-e14f-47e9-91ef-3a4299f00fa6 None None] MFA Rules not processed for user `e600cec7d1694298bd62db0de116adb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:54.979379 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6c7fbee3-e14f-47e9-91ef-3a4299f00fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:55.048815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1092/2169] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:54 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:55.089793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:55.091673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:55.127537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:55.149427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db23cf-c8cc-4ff6-8f40-b9774d36f482 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:55.153120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:55.153753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:55.154035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:55.154311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:55.162584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:55.164014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:55.172654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1078/2170] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:55 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:55.188178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fbee3-e14f-47e9-91ef-3a4299f00fa6 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:55.193599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:55.210335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:55.211775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:55.268826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fbee3-e14f-47e9-91ef-3a4299f00fa6 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:55.273360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:55.274208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:55.274489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:55.274759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] PATH_INFO: `/v3/users/e600cec7d1694298bd62db0de116adb1` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:55.277524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e600cec7d1694298bd62db0de116adb1)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:55.279747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:55.305902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee8e73-c117-46a2-a7b2-eaf5f18bf5f0 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:55.310775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:55.311488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:55.311987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:55.312278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] PATH_INFO: `/v3/projects/cded4b3b0fdd40688ff8cbda6121aa32` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:55.313908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cded4b3b0fdd40688ff8cbda6121aa32)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:55.315374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:55.470585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cded4b3b0fdd40688ff8cbda6121aa32', 'request_id': 'req-c7be54b6-1ed7-4c01-87ce-56127a1edf00'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.471344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e600cec7d1694298bd62db0de116adb1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.472186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e600cec7d1694298bd62db0de116adb1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.496821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cded4b3b0fdd40688ff8cbda6121aa32', 'request_id': 'req-c7be54b6-1ed7-4c01-87ce-56127a1edf00'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.496821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cded4b3b0fdd40688ff8cbda6121aa32', 'request_id': 'req-c7be54b6-1ed7-4c01-87ce-56127a1edf00'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.686182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e600cec7d1694298bd62db0de116adb1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.691951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e600cec7d1694298bd62db0de116adb1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.692656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bae89e42-8e97-4771-8bbf-4407b35f6095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e600cec7d1694298bd62db0de116adb1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.705933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1093/2171] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:55 2022] DELETE /identity/v3/users/e600cec7d1694298bd62db0de116adb1 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:55.714817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:55.715253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:55.715655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:55.735008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:55.740396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:55.761649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] The token cache is being invalidate because project cded4b3b0fdd40688ff8cbda6121aa32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:55.761943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.762598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:55.764357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1079/2172] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:55 2022] DELETE /identity/v3/projects/cded4b3b0fdd40688ff8cbda6121aa32 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:55.768171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:55.769718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:55.801996 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1094/2173] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:55.812187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:55.813635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:55.841476 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:55.842551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be54b6-1ed7-4c01-87ce-56127a1edf00 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:55.847136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:55.847773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:55.848050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:55.848319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] PATH_INFO: `/v3/users/d078d883d5f34714bb233797553a484e` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:55.849543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d078d883d5f34714bb233797553a484e)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:55.852119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:55.857911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:55.870834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f6aed-41dd-4d60-992c-0c17305cda11 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:55.874470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:55.875093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:55.875373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:55.875659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:55.884652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:55.886065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:55.895869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1095/2174] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:14:56.079575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd078d883d5f34714bb233797553a484e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.080303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd078d883d5f34714bb233797553a484e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.157336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd078d883d5f34714bb233797553a484e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.171223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd078d883d5f34714bb233797553a484e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.177923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-996b740a-bb0a-4d53-8be3-abbdf29bfc7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd078d883d5f34714bb233797553a484e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.184235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1080/2175] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:14:55 2022] DELETE /identity/v3/users/d078d883d5f34714bb233797553a484e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:56.280184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:56.283983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:56.294380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc2bc2-c37b-4fe3-8e62-79490d9650bc service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:56.359380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:56.360021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:56.360297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:56.360602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] PATH_INFO: `/v3/projects/6eb915e83f9c42889c5a1359a42cc2c6` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:56.361782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eb915e83f9c42889c5a1359a42cc2c6)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:56.363668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:56.481096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eb915e83f9c42889c5a1359a42cc2c6', 'request_id': 'req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.491639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eb915e83f9c42889c5a1359a42cc2c6', 'request_id': 'req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.520944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eb915e83f9c42889c5a1359a42cc2c6', 'request_id': 'req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.624665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] The token cache is being invalidate because project 6eb915e83f9c42889c5a1359a42cc2c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:56.627049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.627049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.628543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1096/2176] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:56 2022] DELETE /identity/v3/projects/6eb915e83f9c42889c5a1359a42cc2c6 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:56.633914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668387f-a46c-4801-899d-fc7115d5dfad None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:56.635413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668387f-a46c-4801-899d-fc7115d5dfad None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:56.635905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668387f-a46c-4801-899d-fc7115d5dfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:56.659790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7668387f-a46c-4801-899d-fc7115d5dfad None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:56.661359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7668387f-a46c-4801-899d-fc7115d5dfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:56.738337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1081/2177] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:56.746555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:56.746555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:56.760518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650cbc-3583-4c99-a5eb-7271b38a7496 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:56.762493 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650cbc-3583-4c99-a5eb-7271b38a7496 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:56.763041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21650cbc-3583-4c99-a5eb-7271b38a7496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:56.786747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ec034-738e-4de0-bf47-fcd9b28f16c3 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:56.786747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:56.786747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:56.786747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:56.787561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] PATH_INFO: `/v3/projects/16b8dd84d08e4bacb20d2b8b5387407b` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:56.790057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b8dd84d08e4bacb20d2b8b5387407b)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:56.795210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-21650cbc-3583-4c99-a5eb-7271b38a7496 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:56.796785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-21650cbc-3583-4c99-a5eb-7271b38a7496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:56.801254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:56.809188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1082/2178] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:56.825555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21650cbc-3583-4c99-a5eb-7271b38a7496 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:56.841405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21650cbc-3583-4c99-a5eb-7271b38a7496 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:56.846788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:56.847410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:56.847691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:56.847966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:56.848710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:56.853556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:56.853775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:56.869164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1083/2179] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:14:56.873489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:56.890291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9a75-ae42-4b6e-81d1-5c178ab8f68b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:56.895024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:56.895640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:56.895912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:56.896184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:56.896888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:56.898806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:56.935551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b8dd84d08e4bacb20d2b8b5387407b', 'request_id': 'req-940f2c19-9f8a-49f9-becb-2f039bcc97dd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.937626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:56.942232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b8dd84d08e4bacb20d2b8b5387407b', 'request_id': 'req-940f2c19-9f8a-49f9-becb-2f039bcc97dd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:56.948272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b8dd84d08e4bacb20d2b8b5387407b', 'request_id': 'req-940f2c19-9f8a-49f9-becb-2f039bcc97dd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:57.000432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1084/2180] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:56 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:57.004153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.014023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] The token cache is being invalidate because project 16b8dd84d08e4bacb20d2b8b5387407b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:14:57.016190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:57.016190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-940f2c19-9f8a-49f9-becb-2f039bcc97dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:14:57.020489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1097/2181] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:14:56 2022] DELETE /identity/v3/projects/16b8dd84d08e4bacb20d2b8b5387407b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:57.021656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dd41b-3335-4164-afae-88695416321c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.021993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dd41b-3335-4164-afae-88695416321c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.022282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78dd41b-3335-4164-afae-88695416321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.044892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c78dd41b-3335-4164-afae-88695416321c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:57.046324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c78dd41b-3335-4164-afae-88695416321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:57.076518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:57.097230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b89203-28b6-4c9e-ad28-5dd8fa92763f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.103123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.108086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.108086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.108086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.108086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.108086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.113625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1098/2182] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:57.121991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a6cd-bde6-4b1e-ab04-196f7941ec9c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.122361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a6cd-bde6-4b1e-ab04-196f7941ec9c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.122524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a6cd-bde6-4b1e-ab04-196f7941ec9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.141966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3293a6cd-bde6-4b1e-ab04-196f7941ec9c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:57.143163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3293a6cd-bde6-4b1e-ab04-196f7941ec9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:57.151737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1099/2183] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:57.155481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293a6cd-bde6-4b1e-ab04-196f7941ec9c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.165720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293a6cd-bde6-4b1e-ab04-196f7941ec9c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.170962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.171607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.171913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.172027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1085/2184] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:57.172157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.172959 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:14:57.173482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.176029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.177204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.184557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:57.192415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1100/2185] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:14:57.193982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.195641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8d09f-f318-4140-ba8f-79b375a842ee admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.202301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.204540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.204540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.204540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.204784 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12356aa-4eae-49ba-84ba-28a71fb3b6f5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.204868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.206028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.208865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.209538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.209895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.210240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.211005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.213085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.220184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1086/2186] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:57.221269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.234212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2e089-bb33-4280-841f-9460e6d8c3a1 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.239980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.240644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.240927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.241204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.242555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.244466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.255475 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1087/2187] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:57.259629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.269174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9750823-a6a5-4488-99b7-b910f6e991bc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.274027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791d209-78d6-439a-a5d0-6a2341c9dc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.274687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791d209-78d6-439a-a5d0-6a2341c9dc39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.274970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791d209-78d6-439a-a5d0-6a2341c9dc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.275249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791d209-78d6-439a-a5d0-6a2341c9dc39 admin admin] PATH_INFO: `/v3/projects/38335bc4e521473d862606e156feacf3/users/e4e5be32e6134eee870539e866f12da2/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.291723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791d209-78d6-439a-a5d0-6a2341c9dc39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38335bc4e521473d862606e156feacf3, user_id=e4e5be32e6134eee870539e866f12da2, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.294477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791d209-78d6-439a-a5d0-6a2341c9dc39 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.332326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1101/2188] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:57.345997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.411440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1088/2189] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:57 2022] PUT /identity/v3/projects/38335bc4e521473d862606e156feacf3/users/e4e5be32e6134eee870539e866f12da2/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:57.412884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7f7bb-e047-420c-bd27-0cfab386539d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.414532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36133254-9d91-432b-9c07-7328119b26fb None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.415087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36133254-9d91-432b-9c07-7328119b26fb None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.415706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36133254-9d91-432b-9c07-7328119b26fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.454132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-36133254-9d91-432b-9c07-7328119b26fb None None] MFA Rules not processed for user `e4e5be32e6134eee870539e866f12da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:57.456630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-36133254-9d91-432b-9c07-7328119b26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:57.471515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:57.485385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.485774 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.486051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.486321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.486990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.488986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.539078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1089/2190] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:57.550707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36133254-9d91-432b-9c07-7328119b26fb None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.554596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-36133254-9d91-432b-9c07-7328119b26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:57.585111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1102/2191] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:57.591048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.601949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd50d2-4a0a-4d80-97b5-e0cccc95410d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.606817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.607747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.608250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.608553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.609447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.611149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.622012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1103/2192] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:57.632564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.639636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b2c49-6be5-4bf5-b2c5-68b89b6ba2e5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.645007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.645770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.646151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.646429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.647254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.649635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.655264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36133254-9d91-432b-9c07-7328119b26fb None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.660183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc364695-995d-4e7c-8903-28310fd9403c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.661034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc364695-995d-4e7c-8903-28310fd9403c service glance] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.661319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc364695-995d-4e7c-8903-28310fd9403c service glance] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.661432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1104/2193] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:57.661617 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc364695-995d-4e7c-8903-28310fd9403c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.666077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:57.673135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc364695-995d-4e7c-8903-28310fd9403c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.674655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc364695-995d-4e7c-8903-28310fd9403c service glance] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.676852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a89085-0a54-4e65-b188-002394b32585 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:57.679843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:57.683254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:57.683932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.684285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.684513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] PATH_INFO: `/v3/projects/d9b8ca7448b147ce92f90ad17f0bbe03/users/68792ddf4c664b66996981562d4c044c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.685727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1090/2194] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:57.708681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b8ca7448b147ce92f90ad17f0bbe03, user_id=68792ddf4c664b66996981562d4c044c, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:57.708681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:57.801807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1105/2195] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:57 2022] PUT /identity/v3/projects/d9b8ca7448b147ce92f90ad17f0bbe03/users/68792ddf4c664b66996981562d4c044c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:14:57.818855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b3478-e549-4d52-a10c-3e9a0ab720b7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:57.819138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b3478-e549-4d52-a10c-3e9a0ab720b7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:57.819409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b3478-e549-4d52-a10c-3e9a0ab720b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:57.864355 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-549b3478-e549-4d52-a10c-3e9a0ab720b7 None None] MFA Rules not processed for user `68792ddf4c664b66996981562d4c044c`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:57.876001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-549b3478-e549-4d52-a10c-3e9a0ab720b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:57.971341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1091/2196] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:14:57.975751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:58.051280 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f30d0-123f-439f-ad20-a8a99266c24d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:58.056505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:58.057133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.057411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.057682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.058361 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:58.060607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:58.184106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1106/2197] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:14:57 2022] POST /identity/v3/projects => generated 365 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:14:58.185252 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b3478-e549-4d52-a10c-3e9a0ab720b7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:58.196697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b3478-e549-4d52-a10c-3e9a0ab720b7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:58.253446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:58.254070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.254336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.254613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.255292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:58.257424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:58.389574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1092/2198] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:14:58 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:14:58.390751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:58.401145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d601b0-fd7d-4ed2-935f-84712fe62e55 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:58.406749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:58.407410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.407713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.407986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.408723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:58.410458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:58.421059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1107/2199] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:14:58.425517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:58.436270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244b05-4b11-461d-9199-44fb3e5ed701 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:58.440837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:58.441490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.441775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.442056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] PATH_INFO: `/v3/projects/1ce02c1c9e2f418db21c6ac36f11d347/users/9f3aa7638fd14c46852a1517822e7525/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.457302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce02c1c9e2f418db21c6ac36f11d347, user_id=9f3aa7638fd14c46852a1517822e7525, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:58.460078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:58.505690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1093/2200] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:58 2022] PUT /identity/v3/projects/1ce02c1c9e2f418db21c6ac36f11d347/users/9f3aa7638fd14c46852a1517822e7525/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:58.509804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:58.520141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6972644-1232-4292-a664-ae8ffc7aff48 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:58.595869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441217f-70b7-4dcc-9102-28fec6f240e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:58.601926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441217f-70b7-4dcc-9102-28fec6f240e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.601926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441217f-70b7-4dcc-9102-28fec6f240e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.601926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441217f-70b7-4dcc-9102-28fec6f240e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.601926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441217f-70b7-4dcc-9102-28fec6f240e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:58.601926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441217f-70b7-4dcc-9102-28fec6f240e1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:58.605504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:58.610415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1108/2201] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:14:58.614704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:58.626231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88079ad9-5dfe-49e3-9235-c8c99c68a81a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:58.630959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:58.631619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.631948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.632686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] PATH_INFO: `/v3/projects/1ce02c1c9e2f418db21c6ac36f11d347/users/9f3aa7638fd14c46852a1517822e7525/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.634239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce02c1c9e2f418db21c6ac36f11d347, user_id=9f3aa7638fd14c46852a1517822e7525, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:58.636921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:58.647642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1094/2202] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:14:58 2022] PUT /identity/v3/projects/1ce02c1c9e2f418db21c6ac36f11d347/users/9f3aa7638fd14c46852a1517822e7525/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:14:58.657908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba2b2-0e7c-44e9-a0ba-f3730e5037e4 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.660237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba2b2-0e7c-44e9-a0ba-f3730e5037e4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.660237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba2b2-0e7c-44e9-a0ba-f3730e5037e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.688366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-74eba2b2-0e7c-44e9-a0ba-f3730e5037e4 None None] MFA Rules not processed for user `9f3aa7638fd14c46852a1517822e7525`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:58.689558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-74eba2b2-0e7c-44e9-a0ba-f3730e5037e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:58.757816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1109/2203] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:14:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:14:58.769619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:58.771378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:58.818408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:58.829838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d2443-860f-4ac7-8354-6749030f418e admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:58.833616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:58.834226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:58.834509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:58.834789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:58.844504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:58.846555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:58.857221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1095/2204] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:59.092337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba2b2-0e7c-44e9-a0ba-f3730e5037e4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:59.102503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba2b2-0e7c-44e9-a0ba-f3730e5037e4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:59.106299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cf342-fc9f-4c30-adaf-848fce65207d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:59.106982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cf342-fc9f-4c30-adaf-848fce65207d service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:59.107337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cf342-fc9f-4c30-adaf-848fce65207d service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:59.107671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cf342-fc9f-4c30-adaf-848fce65207d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:59.162715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cf342-fc9f-4c30-adaf-848fce65207d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:59.164777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cf342-fc9f-4c30-adaf-848fce65207d service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:59.173710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1110/2205] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:06.998571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:07.000284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:07.100064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5b1b9-6538-4249-b730-5329b35596e3 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:07.105580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:07.106252 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:07.106531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:07.106805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] PATH_INFO: `/v3/users/2f359aa5a3264eafa8eeab4f1701f59b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:07.108004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f359aa5a3264eafa8eeab4f1701f59b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:07.109968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:07.347717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f359aa5a3264eafa8eeab4f1701f59b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:07.348591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f359aa5a3264eafa8eeab4f1701f59b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:07.450434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f359aa5a3264eafa8eeab4f1701f59b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:07.466144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f359aa5a3264eafa8eeab4f1701f59b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:07.473687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f359aa5a3264eafa8eeab4f1701f59b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:07.481250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1096/2206] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:06 2022] DELETE /identity/v3/users/2f359aa5a3264eafa8eeab4f1701f59b => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:07.485624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:07.486141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:07.486247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:07.507796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:07.509138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:07.567651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1111/2207] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:15:07.578194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:07.579600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:07.637007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373a2b-77f6-4a14-b71b-5f01f86fdae3 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:07.640604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:07.641218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:07.641499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:07.641771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:07.650543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:07.651954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:07.661241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1097/2208] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:08.066090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:08.067596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:08.080195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abe720-104b-4a0c-ad91-944b64554f94 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:08.091195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:08.091195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:08.091195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:08.091195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] PATH_INFO: `/v3/projects/db7b2778ff8948f6826d45020132363f` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:08.091665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7b2778ff8948f6826d45020132363f)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:08.091665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:08.115796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:08.210668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7b2778ff8948f6826d45020132363f', 'request_id': 'req-86bba97a-b143-4f5f-8dad-e36bf753de79'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:08.217037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7b2778ff8948f6826d45020132363f', 'request_id': 'req-86bba97a-b143-4f5f-8dad-e36bf753de79'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:08.222617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7b2778ff8948f6826d45020132363f', 'request_id': 'req-86bba97a-b143-4f5f-8dad-e36bf753de79'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:08.304573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] The token cache is being invalidate because project db7b2778ff8948f6826d45020132363f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:08.304796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:08.305422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:08.306939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1112/2209] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:08 2022] DELETE /identity/v3/projects/db7b2778ff8948f6826d45020132363f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:15.795328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:15.797105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:15.811030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb7d4-e7bb-43de-8857-9144d987d3b3 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:15.815003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:15.878945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:15.879713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:15.880069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:15.880513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] PATH_INFO: `/v3/users/68792ddf4c664b66996981562d4c044c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:15.881905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68792ddf4c664b66996981562d4c044c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:15.884489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:15.894265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:15.897614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:15.912781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bba97a-b143-4f5f-8dad-e36bf753de79 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:15.917973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:15.918668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:15.918946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:15.919316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] PATH_INFO: `/v3/users/23d2938afd97422795662ec0581d4fa0` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:15.920638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d2938afd97422795662ec0581d4fa0)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:15.922557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:16.077803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68792ddf4c664b66996981562d4c044c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.078448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68792ddf4c664b66996981562d4c044c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.107636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d2938afd97422795662ec0581d4fa0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.108287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d2938afd97422795662ec0581d4fa0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.138861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68792ddf4c664b66996981562d4c044c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.151460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68792ddf4c664b66996981562d4c044c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.158643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-290308e1-d014-4bb4-bd08-06f392527452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68792ddf4c664b66996981562d4c044c'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.166333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1098/2210] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:15 2022] DELETE /identity/v3/users/68792ddf4c664b66996981562d4c044c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:16.171066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.171354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.171657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.189616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d2938afd97422795662ec0581d4fa0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.192431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:16.193685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:16.203934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d2938afd97422795662ec0581d4fa0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.204704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d2938afd97422795662ec0581d4fa0'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.220981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1113/2211] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:15 2022] DELETE /identity/v3/users/23d2938afd97422795662ec0581d4fa0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:16.263316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1099/2212] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:15:16.272690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:16.274351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:16.338552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8040de3-c2c3-4532-b2c1-9a1be732298e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:16.342240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5eaab-fd62-484c-9565-c5de0fb4eefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:16.342971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5eaab-fd62-484c-9565-c5de0fb4eefb service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.343561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5eaab-fd62-484c-9565-c5de0fb4eefb service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.343728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5eaab-fd62-484c-9565-c5de0fb4eefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.377775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:16.406690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5eaab-fd62-484c-9565-c5de0fb4eefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:16.408173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5eaab-fd62-484c-9565-c5de0fb4eefb service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:16.420555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1114/2213] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:16.506314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:16.508043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:16.522153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668dd5b-0dd4-4a78-b1e4-bf56b0494aed None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:16.529318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:16.530163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.534756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.534756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] PATH_INFO: `/v3/projects/3448cb63d9b741a7956a5fc844820111` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.534756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3448cb63d9b741a7956a5fc844820111)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:16.534756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:16.560741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:16.664032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3448cb63d9b741a7956a5fc844820111', 'request_id': 'req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.670383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3448cb63d9b741a7956a5fc844820111', 'request_id': 'req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.676120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3448cb63d9b741a7956a5fc844820111', 'request_id': 'req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.755183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] The token cache is being invalidate because project 3448cb63d9b741a7956a5fc844820111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:16.755565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.756243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:16.760689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1100/2214] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:16 2022] DELETE /identity/v3/projects/3448cb63d9b741a7956a5fc844820111 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:16.767066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabadc1b-58dc-4b9a-a8ec-696dbc98719b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.767066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabadc1b-58dc-4b9a-a8ec-696dbc98719b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.767066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabadc1b-58dc-4b9a-a8ec-696dbc98719b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.788987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-eabadc1b-58dc-4b9a-a8ec-696dbc98719b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:16.790205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-eabadc1b-58dc-4b9a-a8ec-696dbc98719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:16.852162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:16.852623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:16.864543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1115/2215] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:16.875323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4c37-a3b8-44e2-8b3c-c1addd4e0ba4 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.875323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4c37-a3b8-44e2-8b3c-c1addd4e0ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.875323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4c37-a3b8-44e2-8b3c-c1addd4e0ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.917272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3fde4c37-a3b8-44e2-8b3c-c1addd4e0ba4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:16.919687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3fde4c37-a3b8-44e2-8b3c-c1addd4e0ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:16.935063 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1116/2216] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:16.939288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde4c37-a3b8-44e2-8b3c-c1addd4e0ba4 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:16.951582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde4c37-a3b8-44e2-8b3c-c1addd4e0ba4 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:16.964414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:16.964414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.964414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.964414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.964857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:15:16.964857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:16.964857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:16.969912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964d9e1-2732-4f06-ba73-cdb3e9eb6a03 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:16.975960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1117/2217] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:15:16.976091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:16.976091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.976279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.976454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] PATH_INFO: `/v3/projects/d9b8ca7448b147ce92f90ad17f0bbe03` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.977710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:16.977838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b8ca7448b147ce92f90ad17f0bbe03)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:16.979707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:16.987721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b227-857b-4d74-a1ff-f46bc9eaf656 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:16.992914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:16.993797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:16.994075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:16.994347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:16.995013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:16.996971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:17.117386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b8ca7448b147ce92f90ad17f0bbe03', 'request_id': 'req-d252c71f-bda5-4bff-84cd-4f0faf4679b7'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.124141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b8ca7448b147ce92f90ad17f0bbe03', 'request_id': 'req-d252c71f-bda5-4bff-84cd-4f0faf4679b7'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.130408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b8ca7448b147ce92f90ad17f0bbe03', 'request_id': 'req-d252c71f-bda5-4bff-84cd-4f0faf4679b7'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.141088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1118/2218] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:16 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:15:17.141512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:17.212222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7cba-ad1c-4947-95ea-72a767c51d57 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:17.217056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:17.217682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:17.218014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:17.218295 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:17.218969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:17.221016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:17.293074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] The token cache is being invalidate because project d9b8ca7448b147ce92f90ad17f0bbe03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:17.293367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.294013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.295628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1101/2219] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:16 2022] DELETE /identity/v3/projects/d9b8ca7448b147ce92f90ad17f0bbe03 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:17.299642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:17.302093 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:17.309638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1119/2220] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:17 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:15:17.310735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:17.312429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:17.324187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06f5c2-fdad-4ee6-80dd-6c1562ad3650 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:17.328733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:17.329362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:17.329637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:17.329908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:17.330590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:17.332237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:17.351564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1120/2221] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:15:17.355608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:17.364806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:17.365916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd04df-a08c-44e3-819e-251a1de92064 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:17.372447 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:17.373031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:17.373307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:17.373577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:17.374267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:17.376588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:17.383634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:17.389183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252c71f-bda5-4bff-84cd-4f0faf4679b7 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:17.389632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1121/2222] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:15:17.393942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:17.394064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240938-f447-41c2-84bc-28911233652d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:17.394677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240938-f447-41c2-84bc-28911233652d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:17.394970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240938-f447-41c2-84bc-28911233652d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:17.395247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240938-f447-41c2-84bc-28911233652d admin admin] PATH_INFO: `/v3/users/9f3aa7638fd14c46852a1517822e7525` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:17.396512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240938-f447-41c2-84bc-28911233652d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3aa7638fd14c46852a1517822e7525)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:17.398494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240938-f447-41c2-84bc-28911233652d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:17.403643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb36294-8c20-4211-a76e-edf5a6b46839 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:17.408372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc165-7083-4ec3-b5ef-b5c560810622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:17.409173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc165-7083-4ec3-b5ef-b5c560810622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:17.412065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc165-7083-4ec3-b5ef-b5c560810622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:17.412065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc165-7083-4ec3-b5ef-b5c560810622 admin admin] PATH_INFO: `/v3/projects/a74675480e734145a1692641342c3e99/users/caca4eade01642af9322aa72f7443726/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:17.424598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc165-7083-4ec3-b5ef-b5c560810622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74675480e734145a1692641342c3e99, user_id=caca4eade01642af9322aa72f7443726, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:17.427255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc165-7083-4ec3-b5ef-b5c560810622 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:17.574448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1122/2223] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:17 2022] PUT /identity/v3/projects/a74675480e734145a1692641342c3e99/users/caca4eade01642af9322aa72f7443726/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:17.579509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:17.579792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:17.580062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:17.599843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:17.601943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:17.615239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3aa7638fd14c46852a1517822e7525'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.616083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3aa7638fd14c46852a1517822e7525'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.667316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1123/2224] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:15:17.689766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:17.691167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:17.711598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3aa7638fd14c46852a1517822e7525'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.724329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3aa7638fd14c46852a1517822e7525'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.732204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3aa7638fd14c46852a1517822e7525'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:17.738345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1102/2225] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:17 2022] DELETE /identity/v3/users/9f3aa7638fd14c46852a1517822e7525 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:17.754127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ddc3c-24f4-4332-a2d8-695506aa1e01 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:17.810402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0327c-51f4-44d7-9e95-fba415fc4f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:17.811031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0327c-51f4-44d7-9e95-fba415fc4f44 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:17.811315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0327c-51f4-44d7-9e95-fba415fc4f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:17.811623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0327c-51f4-44d7-9e95-fba415fc4f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:17.813601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-97e0327c-51f4-44d7-9e95-fba415fc4f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:17.857519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:17.877793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0327c-51f4-44d7-9e95-fba415fc4f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:17.879206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0327c-51f4-44d7-9e95-fba415fc4f44 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:17.889525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1124/2226] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:15:18.266040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:18.267660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:18.282180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240938-f447-41c2-84bc-28911233652d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:18.287182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:18.287868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:18.288134 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:18.288976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] PATH_INFO: `/v3/projects/1ce02c1c9e2f418db21c6ac36f11d347` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:18.290207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce02c1c9e2f418db21c6ac36f11d347)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:18.292175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:18.397526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce02c1c9e2f418db21c6ac36f11d347', 'request_id': 'req-8099a4e5-a73d-4123-964d-07be506c8c81'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:18.405750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce02c1c9e2f418db21c6ac36f11d347', 'request_id': 'req-8099a4e5-a73d-4123-964d-07be506c8c81'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:18.411704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce02c1c9e2f418db21c6ac36f11d347', 'request_id': 'req-8099a4e5-a73d-4123-964d-07be506c8c81'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:18.505658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] The token cache is being invalidate because project 1ce02c1c9e2f418db21c6ac36f11d347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:18.505956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:18.506626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8099a4e5-a73d-4123-964d-07be506c8c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:18.509724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1103/2227] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:18 2022] DELETE /identity/v3/projects/1ce02c1c9e2f418db21c6ac36f11d347 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:18.519382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ecb13-b657-4696-9093-514550654e3d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:18.519382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ecb13-b657-4696-9093-514550654e3d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:18.519382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ecb13-b657-4696-9093-514550654e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:18.539010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8d9ecb13-b657-4696-9093-514550654e3d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:18.540232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8d9ecb13-b657-4696-9093-514550654e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:18.617058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1125/2228] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:18.627201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a34be-4fa9-4b8f-9bb3-04dbd98cc9a0 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:18.628592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a34be-4fa9-4b8f-9bb3-04dbd98cc9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:18.629493 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a34be-4fa9-4b8f-9bb3-04dbd98cc9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:18.657431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-189a34be-4fa9-4b8f-9bb3-04dbd98cc9a0 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:18.658791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-189a34be-4fa9-4b8f-9bb3-04dbd98cc9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:18.668233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1104/2229] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:18.673059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ecb13-b657-4696-9093-514550654e3d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:18.685298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ecb13-b657-4696-9093-514550654e3d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:18.690977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:18.691843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:18.692586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:18.693002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:18.694139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:15:18.694563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:18.696291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:18.712042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1126/2230] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:15:18.719888 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a34be-4fa9-4b8f-9bb3-04dbd98cc9a0 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:18.730395 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a34be-4fa9-4b8f-9bb3-04dbd98cc9a0 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:18.736237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:18.737332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:18.737680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:18.738026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:18.738774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:18.740849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:18.880919 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1105/2231] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:18 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:15:18.886035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:18.896642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d29cd7-5188-4ef5-a770-f7c6be354175 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:18.954223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:18.954962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:18.955248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:18.955526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:18.956207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:18.958194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:19.037645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1127/2232] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:18 2022] POST /identity/v3/users => generated 284 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:19.041671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:19.053136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90003724-30c5-4858-88b5-cdaa0263afc5 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:19.064429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:19.064429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.064429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.064429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.064778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:19.064778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:19.074839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1106/2233] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:19.078976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:19.092186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014255f-c0b8-44e5-a65a-e7f7febcebaf admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:19.097046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62574c55-b821-47d8-836c-4ffe9eef1768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:19.097723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62574c55-b821-47d8-836c-4ffe9eef1768 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.097992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62574c55-b821-47d8-836c-4ffe9eef1768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.098272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62574c55-b821-47d8-836c-4ffe9eef1768 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.098985 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62574c55-b821-47d8-836c-4ffe9eef1768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:19.100614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62574c55-b821-47d8-836c-4ffe9eef1768 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:19.111812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1128/2234] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:15:19.117097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:19.124367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:19.128370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c75806-bf73-41b2-bc0f-a61a55989f58 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:19.134003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:19.134672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.134953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.135230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] PATH_INFO: `/v3/projects/328108ea44854905b14492da32cc261a/users/eacf152b231a45f1a8bb9c75c3eb1cdd/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.155097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=328108ea44854905b14492da32cc261a, user_id=eacf152b231a45f1a8bb9c75c3eb1cdd, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:19.157936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:19.278588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1107/2235] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:19 2022] PUT /identity/v3/projects/328108ea44854905b14492da32cc261a/users/eacf152b231a45f1a8bb9c75c3eb1cdd/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:19.284091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ab58d-e77a-483b-87e9-976e06e94853 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.284413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ab58d-e77a-483b-87e9-976e06e94853 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.284686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ab58d-e77a-483b-87e9-976e06e94853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.343293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f59ab58d-e77a-483b-87e9-976e06e94853 None None] MFA Rules not processed for user `eacf152b231a45f1a8bb9c75c3eb1cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:19.345228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f59ab58d-e77a-483b-87e9-976e06e94853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:19.402806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:19.441145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1129/2236] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:19.446641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:19.531832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41fc7-1c7f-4360-9a84-842c015bf59d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:19.539130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:19.540250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.540250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.540701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.541521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:19.543542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:19.688940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1108/2237] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:19 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:19.692879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ab58d-e77a-483b-87e9-976e06e94853 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:19.705356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ab58d-e77a-483b-87e9-976e06e94853 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:19.770602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:19.771236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.771519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.771791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.772517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:19.774501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:19.872951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1130/2238] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:19 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:15:19.874440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:19.889211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894faaff-633d-494e-84ff-daaf4cebb9a3 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:19.893790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:19.894410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.894721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.898914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.898914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:19.898914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:19.912450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1109/2239] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:15:19.916344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:19.927209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dd16c-79a8-414b-a092-76ffcd063bec admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:19.932043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:19.933121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:19.933121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:19.933322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] PATH_INFO: `/v3/projects/facb3ddb14e74216aaa5a4e337c12712/users/10c9f137a599461eb4671f3365d104bd/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:19.949546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] RBAC: Authorizing `identity:create_grant(project_id=facb3ddb14e74216aaa5a4e337c12712, user_id=10c9f137a599461eb4671f3365d104bd, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:19.952215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:20.031608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1131/2240] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:19 2022] PUT /identity/v3/projects/facb3ddb14e74216aaa5a4e337c12712/users/10c9f137a599461eb4671f3365d104bd/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:20.036050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:20.047967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0e0fb-44fd-4cf8-b7fd-e731b325c1fc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:20.155337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b36219-c69d-43f8-b501-020e4dcfefef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:20.156084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36219-c69d-43f8-b501-020e4dcfefef admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:20.156476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36219-c69d-43f8-b501-020e4dcfefef admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:20.156824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36219-c69d-43f8-b501-020e4dcfefef admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:20.157601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b36219-c69d-43f8-b501-020e4dcfefef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:20.159388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b36219-c69d-43f8-b501-020e4dcfefef admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:20.184159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1110/2241] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:15:20.188517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:20.201008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4641b-4b48-43eb-9c50-641883ae110b admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:20.207979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:20.208803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:20.209149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:20.209489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] PATH_INFO: `/v3/projects/facb3ddb14e74216aaa5a4e337c12712/users/10c9f137a599461eb4671f3365d104bd/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:20.212943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=facb3ddb14e74216aaa5a4e337c12712, user_id=10c9f137a599461eb4671f3365d104bd, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:20.216083 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:20.231096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1132/2242] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:20 2022] PUT /identity/v3/projects/facb3ddb14e74216aaa5a4e337c12712/users/10c9f137a599461eb4671f3365d104bd/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:20.241985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d41d-1d47-41b6-b646-1653ae765992 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:20.242338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d41d-1d47-41b6-b646-1653ae765992 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:20.242711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7d41d-1d47-41b6-b646-1653ae765992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:20.272935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9fc7d41d-1d47-41b6-b646-1653ae765992 None None] MFA Rules not processed for user `10c9f137a599461eb4671f3365d104bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:20.274196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9fc7d41d-1d47-41b6-b646-1653ae765992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:20.321687 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:20.394096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1111/2243] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:20.409537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:20.411562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:20.503316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e53579-fc1a-4ca6-9dfe-b73f074c74bf admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:20.527245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:20.527896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:20.528250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:20.528562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:20.542619 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:20.544144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:20.555816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1133/2244] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:24.089726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:24.090231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:24.090341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:24.121552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] MFA Rules not processed for user `caca4eade01642af9322aa72f7443726`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:24.122559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:24.166957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:24.213777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:24.226082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1112/2245] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:24.240314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:24.250250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:24.252318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7ca32-e945-428c-b503-539f1635f85e service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:24.261637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:24.264597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:24.265351 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:24.265631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] PATH_INFO: `/v3/users/eacf152b231a45f1a8bb9c75c3eb1cdd` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:24.269142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eacf152b231a45f1a8bb9c75c3eb1cdd)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:24.271099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:24.362921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f75187-b352-4479-bb27-1cbf09c462b5 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:24.374573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab790fa-0ebf-414f-b49d-8be5db334d0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:24.375220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab790fa-0ebf-414f-b49d-8be5db334d0b service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:24.375505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab790fa-0ebf-414f-b49d-8be5db334d0b service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:24.375782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab790fa-0ebf-414f-b49d-8be5db334d0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:24.387590 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab790fa-0ebf-414f-b49d-8be5db334d0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:24.389589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab790fa-0ebf-414f-b49d-8be5db334d0b service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:24.399771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1113/2246] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:15:24.500220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eacf152b231a45f1a8bb9c75c3eb1cdd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:24.500942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eacf152b231a45f1a8bb9c75c3eb1cdd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:24.606915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eacf152b231a45f1a8bb9c75c3eb1cdd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:24.621013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eacf152b231a45f1a8bb9c75c3eb1cdd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:24.621729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eacf152b231a45f1a8bb9c75c3eb1cdd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:24.635244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1134/2247] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:24 2022] DELETE /identity/v3/users/eacf152b231a45f1a8bb9c75c3eb1cdd => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:24.641243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:24.641243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:24.641243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:24.660573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:24.662082 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:24.720459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:24.735956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1114/2248] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:24.747076 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:24.749442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:24.821017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b3996-079c-4c00-8240-89d8de3b530a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:24.827017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb712b-3f94-403a-ac2d-f5a34ae54204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:24.829435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb712b-3f94-403a-ac2d-f5a34ae54204 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:24.829435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb712b-3f94-403a-ac2d-f5a34ae54204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:24.829435 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb712b-3f94-403a-ac2d-f5a34ae54204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:24.843618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb712b-3f94-403a-ac2d-f5a34ae54204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:24.845118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb712b-3f94-403a-ac2d-f5a34ae54204 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:24.859843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1135/2249] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:24.888358 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:24.890713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:24.904623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc54e93-b31f-4634-9339-a14936e99ac1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:24.914510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:24.915254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:24.915734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:24.916035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] PATH_INFO: `/v3/users/ce65af9e391841bb8f077ef49a115115` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:24.917517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce65af9e391841bb8f077ef49a115115)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:24.919965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:25.195746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce65af9e391841bb8f077ef49a115115'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.198195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce65af9e391841bb8f077ef49a115115'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.293786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce65af9e391841bb8f077ef49a115115'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.309008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce65af9e391841bb8f077ef49a115115'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.321646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce65af9e391841bb8f077ef49a115115'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.329333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1115/2250] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:24 2022] DELETE /identity/v3/users/ce65af9e391841bb8f077ef49a115115 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:25.356757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:25.356757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:25.356757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:25.375578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:25.376907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:25.382604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:25.382943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:25.443545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:25.465949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1136/2251] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:15:25.489488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:25.492514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:25.501596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38c88f-84f4-4dd0-a0ce-837fa2ae193b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:25.508149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:25.509501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:25.510733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:25.510733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] PATH_INFO: `/v3/projects/328108ea44854905b14492da32cc261a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:25.512803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=328108ea44854905b14492da32cc261a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:25.515123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:25.588537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f2901-1312-45dd-af07-f2d72a3f76a1 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:25.593891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:25.594465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:25.594976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:25.601276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:25.607490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:25.609504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:25.623089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1137/2252] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:25.703482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328108ea44854905b14492da32cc261a', 'request_id': 'req-14c17a97-3665-416a-822d-02a9b389f617'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.710054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328108ea44854905b14492da32cc261a', 'request_id': 'req-14c17a97-3665-416a-822d-02a9b389f617'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.718286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328108ea44854905b14492da32cc261a', 'request_id': 'req-14c17a97-3665-416a-822d-02a9b389f617'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.720150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:25.865613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] The token cache is being invalidate because project 328108ea44854905b14492da32cc261a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:25.866371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.867311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:25.870628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1116/2253] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:25 2022] DELETE /identity/v3/projects/328108ea44854905b14492da32cc261a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:25.883031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:25.894524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfa849-c8b2-48f5-8d42-f4dd4461b193 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:25.977265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:25.978022 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:25.978834 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:25.979196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] PATH_INFO: `/v3/users/10c9f137a599461eb4671f3365d104bd` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:25.981052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c9f137a599461eb4671f3365d104bd)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:25.984545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:26.172599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c9f137a599461eb4671f3365d104bd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.173523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c9f137a599461eb4671f3365d104bd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.189871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:26.191342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:26.205218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c17a97-3665-416a-822d-02a9b389f617 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:26.216651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:26.216651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:26.216651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:26.216651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] PATH_INFO: `/v3/projects/0b9c0064463b49fdb26e5c6dbcee79fb` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:26.217108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9c0064463b49fdb26e5c6dbcee79fb)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:26.218364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:26.266640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c9f137a599461eb4671f3365d104bd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.283803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c9f137a599461eb4671f3365d104bd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.285149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-429dc03d-ccf7-41dd-b11b-bf02782666a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c9f137a599461eb4671f3365d104bd'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.308607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1138/2254] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:25 2022] DELETE /identity/v3/users/10c9f137a599461eb4671f3365d104bd => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:26.361761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9c0064463b49fdb26e5c6dbcee79fb', 'request_id': 'req-b54dbcb2-c0bd-4702-896a-eb2845c69cea'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.370526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9c0064463b49fdb26e5c6dbcee79fb', 'request_id': 'req-b54dbcb2-c0bd-4702-896a-eb2845c69cea'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.377242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9c0064463b49fdb26e5c6dbcee79fb', 'request_id': 'req-b54dbcb2-c0bd-4702-896a-eb2845c69cea'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.745443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] The token cache is being invalidate because project 0b9c0064463b49fdb26e5c6dbcee79fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:26.745670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.746259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b54dbcb2-c0bd-4702-896a-eb2845c69cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:26.747872 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1117/2255] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:26 2022] DELETE /identity/v3/projects/0b9c0064463b49fdb26e5c6dbcee79fb => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:26.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74ffb1-c63c-45c4-b30f-df811106d633 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:26.753414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74ffb1-c63c-45c4-b30f-df811106d633 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:26.753515 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74ffb1-c63c-45c4-b30f-df811106d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:26.774659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3e74ffb1-c63c-45c4-b30f-df811106d633 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:26.775908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3e74ffb1-c63c-45c4-b30f-df811106d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:26.785073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:26.904290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1139/2256] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:26.904744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca977c-18b1-440c-9d40-f96fd58061ee None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:26.905038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca977c-18b1-440c-9d40-f96fd58061ee None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:26.905317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca977c-18b1-440c-9d40-f96fd58061ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:26.948046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-8eca977c-18b1-440c-9d40-f96fd58061ee None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:26.952004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8eca977c-18b1-440c-9d40-f96fd58061ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:26.979961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74ffb1-c63c-45c4-b30f-df811106d633 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:26.984788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1118/2257] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:26.993444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74ffb1-c63c-45c4-b30f-df811106d633 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.008337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.008337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.008337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.008337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.008866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:15:27.008866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.009854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.034551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1140/2258] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:15:27.039319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca977c-18b1-440c-9d40-f96fd58061ee None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:27.056010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca977c-18b1-440c-9d40-f96fd58061ee None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.068507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.068983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.069802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.070077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.070763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.077129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.176788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:27.179466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:27.187760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1119/2259] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:27 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:27.188876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:27.218925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e0e24-3019-42d5-bac6-6eafbe976d98 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.316055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81250299-9238-44f3-a105-acf90b1c19e6 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.340360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.341028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.341374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.341597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.342284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.344216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.387723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.391102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.391646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.392113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] PATH_INFO: `/v3/projects/facb3ddb14e74216aaa5a4e337c12712` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.400111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=facb3ddb14e74216aaa5a4e337c12712)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.405194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.434608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1120/2260] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:27 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:27.441537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:27.471582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c9a7bd-dc6c-4193-80ba-99d73d71ff96 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.488688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.489535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.489975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.490256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.491498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.493720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.512829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1121/2261] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:15:27.516636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:27.529891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe9d33-9c33-415c-8982-21865977c6cc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.549227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.550149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.550623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.551055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.558785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.562372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.579396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1122/2262] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:27.583669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:27.598945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'facb3ddb14e74216aaa5a4e337c12712', 'request_id': 'req-67792f20-a7e5-4ef8-ac23-c00c16f1f401'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:27.601164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ae8d-b4ab-408b-9677-6640f494d439 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.612461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=6,keystone:DELETE=8,keystone:SELECT=86 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:27.613471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'facb3ddb14e74216aaa5a4e337c12712', 'request_id': 'req-67792f20-a7e5-4ef8-ac23-c00c16f1f401'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:27.625015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'facb3ddb14e74216aaa5a4e337c12712', 'request_id': 'req-67792f20-a7e5-4ef8-ac23-c00c16f1f401'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:27.679235 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db1425-728e-40c1-9461-77cdb6764155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.680007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db1425-728e-40c1-9461-77cdb6764155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.680356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db1425-728e-40c1-9461-77cdb6764155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.680789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db1425-728e-40c1-9461-77cdb6764155 admin admin] PATH_INFO: `/v3/projects/370ac8b4f98e45f5940e643b3afa6be3/users/412828a7f19c451a9784f370f84c37ec/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.702898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db1425-728e-40c1-9461-77cdb6764155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370ac8b4f98e45f5940e643b3afa6be3, user_id=412828a7f19c451a9784f370f84c37ec, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.705795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db1425-728e-40c1-9461-77cdb6764155 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.716668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] The token cache is being invalidate because project facb3ddb14e74216aaa5a4e337c12712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:27.718352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:27.718352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-67792f20-a7e5-4ef8-ac23-c00c16f1f401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:27.719475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1141/2263] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:27 2022] DELETE /identity/v3/projects/facb3ddb14e74216aaa5a4e337c12712 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:27.725894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e55ae7-740b-4754-8fc4-94a852e065b4 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.726187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e55ae7-740b-4754-8fc4-94a852e065b4 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.726470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e55ae7-740b-4754-8fc4-94a852e065b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.749588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-73e55ae7-740b-4754-8fc4-94a852e065b4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:27.750810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-73e55ae7-740b-4754-8fc4-94a852e065b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:27.758967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1142/2264] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:27.767044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3bd61-b5f1-44f5-9d36-be7dbae4a685 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.767237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3bd61-b5f1-44f5-9d36-be7dbae4a685 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.767416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3bd61-b5f1-44f5-9d36-be7dbae4a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.801396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-acd3bd61-b5f1-44f5-9d36-be7dbae4a685 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:27.802942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-acd3bd61-b5f1-44f5-9d36-be7dbae4a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:27.828010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1123/2265] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:27 2022] PUT /identity/v3/projects/370ac8b4f98e45f5940e643b3afa6be3/users/412828a7f19c451a9784f370f84c37ec/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:27.843329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1143/2266] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:27.846538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557abe0-6cba-472f-8b91-822fe8f10807 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.846808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557abe0-6cba-472f-8b91-822fe8f10807 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.846896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3bd61-b5f1-44f5-9d36-be7dbae4a685 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:27.846977 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557abe0-6cba-472f-8b91-822fe8f10807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.860600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3bd61-b5f1-44f5-9d36-be7dbae4a685 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:27.883739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:27.912686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3557abe0-6cba-472f-8b91-822fe8f10807 None None] MFA Rules not processed for user `412828a7f19c451a9784f370f84c37ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:27.915102 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3557abe0-6cba-472f-8b91-822fe8f10807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:27.967119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:27.967119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:27.967119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:27.967119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:27.967518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:15:27.967935 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:27.969605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:27.986140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1144/2267] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:15:27.998664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:28.005044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292829e1-9839-4e99-b947-cf75711cb2a1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:28.012347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:28.013521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.013977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.014285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.015033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:28.017213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:28.026452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1124/2268] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:28.028264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b10e7-26fb-473c-8b31-e8b10f1e9fe9 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.029040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b10e7-26fb-473c-8b31-e8b10f1e9fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.029164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b10e7-26fb-473c-8b31-e8b10f1e9fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.051316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-724b10e7-26fb-473c-8b31-e8b10f1e9fe9 None None] MFA Rules not processed for user `caca4eade01642af9322aa72f7443726`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:28.054243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-724b10e7-26fb-473c-8b31-e8b10f1e9fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:28.108497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1145/2269] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:27 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:28.108497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:28.110427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:28.124521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1125/2270] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:28.125435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b10e7-26fb-473c-8b31-e8b10f1e9fe9 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:28.141588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b10e7-26fb-473c-8b31-e8b10f1e9fe9 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:28.191365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9411e-521f-41c1-ade7-4fc11ccc8884 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:28.201147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:28.204860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.204860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.204860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.222781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:28.223914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.224354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.224906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.225835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:28.227972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:28.290564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:28.292015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:28.302914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1146/2271] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:28.352173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1126/2272] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:28 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:28.357203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:28.374512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ca07e-f1b5-4863-96eb-1f79ae3743bd service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:28.381011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:28.381941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.382391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.382713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.383920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:28.386026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:28.405452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1147/2273] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:28.410433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:28.430383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb961ef0-041b-45b2-a0f0-95172ab5dc12 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:28.440976 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e82556-6aaf-4919-beee-df73990f14c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:28.444159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e82556-6aaf-4919-beee-df73990f14c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.444159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e82556-6aaf-4919-beee-df73990f14c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.444159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e82556-6aaf-4919-beee-df73990f14c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.444159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e82556-6aaf-4919-beee-df73990f14c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:28.449441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e82556-6aaf-4919-beee-df73990f14c8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:28.470822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1127/2274] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:28.474639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:28.486002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ccd13-8793-4af6-bb7d-940f3c0c8d25 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:28.493882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:28.494670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.495028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.495384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] PATH_INFO: `/v3/projects/5f422eb58af24d25a4b54c0af48e2bd3/users/7ae5a9c6137c4ca98afa136029c8961d/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.523552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f422eb58af24d25a4b54c0af48e2bd3, user_id=7ae5a9c6137c4ca98afa136029c8961d, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:28.527625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:28.618568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1148/2275] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:28 2022] PUT /identity/v3/projects/5f422eb58af24d25a4b54c0af48e2bd3/users/7ae5a9c6137c4ca98afa136029c8961d/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:28.621897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ca457-bc47-4745-b47a-9343e74ae7a2 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.622197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ca457-bc47-4745-b47a-9343e74ae7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.622479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ca457-bc47-4745-b47a-9343e74ae7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.662422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1f6ca457-bc47-4745-b47a-9343e74ae7a2 None None] MFA Rules not processed for user `7ae5a9c6137c4ca98afa136029c8961d`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:28.663713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1f6ca457-bc47-4745-b47a-9343e74ae7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:28.686053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:28.779629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1128/2276] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:28.791635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:28.793921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:28.808081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:28.898074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05b918-53b2-4d2e-904a-f0c22948f219 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:28.907061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:28.908003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:28.908452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:28.908871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:28.919137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:28.920691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:28.940536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1149/2277] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:30.159740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136f6ee-1bf2-4b1e-ae64-a3bbf85cfdaf None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:30.160257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136f6ee-1bf2-4b1e-ae64-a3bbf85cfdaf None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:30.160257 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136f6ee-1bf2-4b1e-ae64-a3bbf85cfdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:30.179328 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9136f6ee-1bf2-4b1e-ae64-a3bbf85cfdaf None None] MFA Rules not processed for user `412828a7f19c451a9784f370f84c37ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:30.181286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9136f6ee-1bf2-4b1e-ae64-a3bbf85cfdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:30.239719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1129/2278] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:39.104890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=22 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:40.232631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:44.491120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:44.579603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa430e45-03a9-4403-8325-37ec5befc27b service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:44.584508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:44.585891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:44.588372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:44.588372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] PATH_INFO: `/v3/users/7ae5a9c6137c4ca98afa136029c8961d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:44.588372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae5a9c6137c4ca98afa136029c8961d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:44.590067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:44.850265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae5a9c6137c4ca98afa136029c8961d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:44.851029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae5a9c6137c4ca98afa136029c8961d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:44.938722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae5a9c6137c4ca98afa136029c8961d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:44.953073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae5a9c6137c4ca98afa136029c8961d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:44.953957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae5a9c6137c4ca98afa136029c8961d'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:44.966563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1150/2279] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:44 2022] DELETE /identity/v3/users/7ae5a9c6137c4ca98afa136029c8961d => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:44.971005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:44.971568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:44.971659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:44.992270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:44.993775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:45.072526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1130/2280] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:45.081798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:45.096265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:45.149135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaab549-efd3-4d87-8a44-b7eacd194e60 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:45.152769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d06aa3-60dc-40e9-8f72-b33a42e7a892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:45.153412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06aa3-60dc-40e9-8f72-b33a42e7a892 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:45.153714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06aa3-60dc-40e9-8f72-b33a42e7a892 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:45.153962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06aa3-60dc-40e9-8f72-b33a42e7a892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:45.163489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d06aa3-60dc-40e9-8f72-b33a42e7a892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:45.164974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d06aa3-60dc-40e9-8f72-b33a42e7a892 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:45.174432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1151/2281] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:45.579731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:45.581231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:45.594215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972ef6b-7110-467b-b56d-9d7069db33af None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:45.598921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:45.599696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:45.599988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:45.600260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] PATH_INFO: `/v3/projects/5f422eb58af24d25a4b54c0af48e2bd3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:45.602019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f422eb58af24d25a4b54c0af48e2bd3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:45.614662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:45.734596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f422eb58af24d25a4b54c0af48e2bd3', 'request_id': 'req-2c23b97a-764b-4905-9310-4185f38a7cb2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:45.740810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f422eb58af24d25a4b54c0af48e2bd3', 'request_id': 'req-2c23b97a-764b-4905-9310-4185f38a7cb2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:45.748160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f422eb58af24d25a4b54c0af48e2bd3', 'request_id': 'req-2c23b97a-764b-4905-9310-4185f38a7cb2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:45.823802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] The token cache is being invalidate because project 5f422eb58af24d25a4b54c0af48e2bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:45.824160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:45.824818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c23b97a-764b-4905-9310-4185f38a7cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:45.826330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1131/2282] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:45 2022] DELETE /identity/v3/projects/5f422eb58af24d25a4b54c0af48e2bd3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:45.831305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28ee8-e8de-49e9-9dec-61173a2aabda None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:45.831794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28ee8-e8de-49e9-9dec-61173a2aabda None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:45.831962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca28ee8-e8de-49e9-9dec-61173a2aabda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:45.854694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bca28ee8-e8de-49e9-9dec-61173a2aabda None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:45.856611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bca28ee8-e8de-49e9-9dec-61173a2aabda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:45.881231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:45.936491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1152/2283] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:45.945974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062b41-e0e0-4350-b8e0-178677d63226 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:45.946341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062b41-e0e0-4350-b8e0-178677d63226 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:45.946684 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062b41-e0e0-4350-b8e0-178677d63226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:45.970026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c0062b41-e0e0-4350-b8e0-178677d63226 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:45.971279 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c0062b41-e0e0-4350-b8e0-178677d63226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:45.984508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1132/2284] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:45.986149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca28ee8-e8de-49e9-9dec-61173a2aabda None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:46.008856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca28ee8-e8de-49e9-9dec-61173a2aabda None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:46.016592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:46.020786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.020786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.020786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.020786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:15:46.020786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:46.020786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:46.039956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1153/2285] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:15:46.048874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0062b41-e0e0-4350-b8e0-178677d63226 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:46.064113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0062b41-e0e0-4350-b8e0-178677d63226 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:46.075550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:46.077054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.078139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.078438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.079136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:46.091829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:46.268573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1133/2286] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:46 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:46.272984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:46.283775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b18f67-b254-4779-ad3e-2a5c0316f0fb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:46.359654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:46.363159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.363159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.363159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.363159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:46.365037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:46.473779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1154/2287] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:46 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:15:46.476073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:46.487968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3891726-4d8a-47dd-8244-67b626a70678 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:46.501110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:46.501834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.502194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.502620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.503408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:46.505210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:46.515785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1134/2288] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:46.519721 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:46.530158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad062c27-4c30-4249-85aa-3b1dfec67a81 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:46.536055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4ea73-124a-42ae-a3a0-cbe551181696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:46.536792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ea73-124a-42ae-a3a0-cbe551181696 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.537070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ea73-124a-42ae-a3a0-cbe551181696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.538655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ea73-124a-42ae-a3a0-cbe551181696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.539374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4ea73-124a-42ae-a3a0-cbe551181696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:46.541423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4ea73-124a-42ae-a3a0-cbe551181696 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:46.552747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1155/2289] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:46.557536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:46.567559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2d359-594b-49a3-b4f2-a50202a177de admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:46.572336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:46.573021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.573307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.573602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] PATH_INFO: `/v3/projects/c0331f7668124d02b846862db18ed534/users/e759a0fed72d4524b785c4f443d3a0de/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.588869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0331f7668124d02b846862db18ed534, user_id=e759a0fed72d4524b785c4f443d3a0de, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:46.591532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:46.745555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1135/2290] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:46 2022] PUT /identity/v3/projects/c0331f7668124d02b846862db18ed534/users/e759a0fed72d4524b785c4f443d3a0de/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:46.756200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa78b-41b6-413b-9612-ad0ed4110646 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.756525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa78b-41b6-413b-9612-ad0ed4110646 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.756808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa78b-41b6-413b-9612-ad0ed4110646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.786128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5edaa78b-41b6-413b-9612-ad0ed4110646 None None] MFA Rules not processed for user `e759a0fed72d4524b785c4f443d3a0de`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:46.787627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5edaa78b-41b6-413b-9612-ad0ed4110646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:46.863114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1156/2291] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:46.876113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:46.877963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:46.900816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:46.936254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c093749-2a54-4406-8fc9-9901becd97da admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:46.940037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421b191-9d9e-41bc-bcb0-0a0031a0970a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:46.940624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421b191-9d9e-41bc-bcb0-0a0031a0970a service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:46.940904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421b191-9d9e-41bc-bcb0-0a0031a0970a service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:46.941177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421b191-9d9e-41bc-bcb0-0a0031a0970a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:46.950216 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421b191-9d9e-41bc-bcb0-0a0031a0970a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:46.951722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421b191-9d9e-41bc-bcb0-0a0031a0970a service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:46.961728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1136/2292] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:52.178005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa78b-41b6-413b-9612-ad0ed4110646 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:52.198499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:52.241728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa78b-41b6-413b-9612-ad0ed4110646 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:52.247201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:52.247819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:52.248099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:52.248451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] PATH_INFO: `/v3/users/e759a0fed72d4524b785c4f443d3a0de` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:52.249618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e759a0fed72d4524b785c4f443d3a0de)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:52.251615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:52.426555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e759a0fed72d4524b785c4f443d3a0de'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:52.427331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e759a0fed72d4524b785c4f443d3a0de'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:52.509929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e759a0fed72d4524b785c4f443d3a0de'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:52.522862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e759a0fed72d4524b785c4f443d3a0de'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:52.523558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e759a0fed72d4524b785c4f443d3a0de'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:52.536694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1157/2293] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:52 2022] DELETE /identity/v3/users/e759a0fed72d4524b785c4f443d3a0de => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:52.541771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:52.542246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:52.542643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:52.581913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:52.581913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:52.657019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1137/2294] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:15:52.669653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:52.671294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:52.736524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a16fe-2b5c-4ffd-ab8a-87d79bc002fe admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:52.740139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7ed21-02fd-440c-b8d9-e91b66223133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:52.740788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7ed21-02fd-440c-b8d9-e91b66223133 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:52.741071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7ed21-02fd-440c-b8d9-e91b66223133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:52.741345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7ed21-02fd-440c-b8d9-e91b66223133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:52.751472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7ed21-02fd-440c-b8d9-e91b66223133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:52.753121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7ed21-02fd-440c-b8d9-e91b66223133 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:52.765336 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1158/2295] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:53.237566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:53.239011 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:53.251566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8ad17-2298-46ef-8e9b-e9a3e6ea134b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:53.258281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:53.258939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.259107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.259386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] PATH_INFO: `/v3/projects/c0331f7668124d02b846862db18ed534` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.260782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0331f7668124d02b846862db18ed534)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:53.262740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:53.380771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0331f7668124d02b846862db18ed534', 'request_id': 'req-f39b9b06-65fc-40b8-8e63-00fed3886d44'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:53.387789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0331f7668124d02b846862db18ed534', 'request_id': 'req-f39b9b06-65fc-40b8-8e63-00fed3886d44'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:53.393438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0331f7668124d02b846862db18ed534', 'request_id': 'req-f39b9b06-65fc-40b8-8e63-00fed3886d44'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:53.463662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] The token cache is being invalidate because project c0331f7668124d02b846862db18ed534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:53.463952 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:53.464611 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f39b9b06-65fc-40b8-8e63-00fed3886d44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:53.466148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1138/2296] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:53 2022] DELETE /identity/v3/projects/c0331f7668124d02b846862db18ed534 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:53.471041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9931ee-c7d1-443c-99fb-9f003f59a700 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.471494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9931ee-c7d1-443c-99fb-9f003f59a700 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.471705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9931ee-c7d1-443c-99fb-9f003f59a700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.491042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-7e9931ee-c7d1-443c-99fb-9f003f59a700 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:53.492246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7e9931ee-c7d1-443c-99fb-9f003f59a700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:53.527213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:53.554227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1159/2297] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:53.562697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06fdb9-ccfe-46e4-aa22-7126edbd13f6 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.563181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06fdb9-ccfe-46e4-aa22-7126edbd13f6 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.563303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06fdb9-ccfe-46e4-aa22-7126edbd13f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.583609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2c06fdb9-ccfe-46e4-aa22-7126edbd13f6 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:53.584855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2c06fdb9-ccfe-46e4-aa22-7126edbd13f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:53.593383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1139/2298] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:53.597752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9931ee-c7d1-443c-99fb-9f003f59a700 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:53.609169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9931ee-c7d1-443c-99fb-9f003f59a700 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:53.613921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:53.614542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.615527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.615822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.616554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:15:53.617348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:53.619190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:53.630953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1160/2299] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:15:53.636057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06fdb9-ccfe-46e4-aa22-7126edbd13f6 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:53.651019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06fdb9-ccfe-46e4-aa22-7126edbd13f6 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:53.657943 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:53.658758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.658874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.659540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.659823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:53.661810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:53.776164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1140/2300] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:53 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:53.777445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:53.787506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70763b-57ce-4f07-8983-ad21c03c2b86 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:53.846711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:53.847343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.848279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.848279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.848612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:53.850587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:53.922118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1161/2301] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:53 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:53.926236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:53.937114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addc454-0c40-497b-99fd-e19b18856435 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:53.941809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:53.942443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.942694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.942970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.943669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:53.945343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:53.955076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1141/2302] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:53.958988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:53.969563 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91611-dd50-4925-8eec-c88f0f947e63 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:53.975375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6dde5-9d53-4e15-b2b0-8a8486e01c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:53.975994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6dde5-9d53-4e15-b2b0-8a8486e01c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:53.976838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6dde5-9d53-4e15-b2b0-8a8486e01c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:53.977118 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6dde5-9d53-4e15-b2b0-8a8486e01c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:53.977824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6dde5-9d53-4e15-b2b0-8a8486e01c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:53.979417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6dde5-9d53-4e15-b2b0-8a8486e01c7b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:53.991366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1162/2303] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:53.996318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:54.006353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463fb81-560c-48ba-bc5f-d859975ca044 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:54.011005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:54.011660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:54.011941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:54.012219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] PATH_INFO: `/v3/projects/07f2f5cb5bcb4342a5f798d470a96cd0/users/58c183e928a1454a97143d4292ae2cf3/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:54.014501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:54.027087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f2f5cb5bcb4342a5f798d470a96cd0, user_id=58c183e928a1454a97143d4292ae2cf3, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:54.029816 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:54.132894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1142/2304] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:53 2022] PUT /identity/v3/projects/07f2f5cb5bcb4342a5f798d470a96cd0/users/58c183e928a1454a97143d4292ae2cf3/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:54.144576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:54.144780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:54.144998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:54.177539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] MFA Rules not processed for user `58c183e928a1454a97143d4292ae2cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:54.178951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:54.274430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1163/2305] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:54.288551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:54.291383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:54.353439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eff3d-8693-4b9b-8819-e13f390fb826 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:54.357168 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f316edb-a2e8-4d21-b02b-d8fccb0892c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:54.357784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f316edb-a2e8-4d21-b02b-d8fccb0892c3 service glance] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:54.358065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f316edb-a2e8-4d21-b02b-d8fccb0892c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:54.358341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f316edb-a2e8-4d21-b02b-d8fccb0892c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:54.366856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f316edb-a2e8-4d21-b02b-d8fccb0892c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:54.368593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f316edb-a2e8-4d21-b02b-d8fccb0892c3 service glance] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:54.377562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1143/2306] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:57.031742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:57.033481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:57.093066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:57.124145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58ea42-b715-4b4b-817e-97a93ecbe3ac None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:57.129930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:57.130698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:57.131084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:57.131509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] PATH_INFO: `/v3/users/e4e5be32e6134eee870539e866f12da2` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:57.133545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e5be32e6134eee870539e866f12da2)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:57.136867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:57.599918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e5be32e6134eee870539e866f12da2'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:57.600628 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e5be32e6134eee870539e866f12da2'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:57.703306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e5be32e6134eee870539e866f12da2'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:57.718136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e5be32e6134eee870539e866f12da2'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:57.718909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e5be32e6134eee870539e866f12da2'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:57.733865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1164/2307] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:15:57 2022] DELETE /identity/v3/users/e4e5be32e6134eee870539e866f12da2 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:15:57.738450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:57.738994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:57.738994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:57.766367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:57.766367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:57.838565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1144/2308] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:57.848748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:57.850254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:57.911240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b81ef-99b9-4e77-b7ee-6c9b7e25db10 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:57.916576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1b121-2b11-416c-b6ad-1ce0d72513ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:57.916799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1b121-2b11-416c-b6ad-1ce0d72513ad service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:57.917053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1b121-2b11-416c-b6ad-1ce0d72513ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:57.917307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1b121-2b11-416c-b6ad-1ce0d72513ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:57.928570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1b121-2b11-416c-b6ad-1ce0d72513ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:57.930602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1b121-2b11-416c-b6ad-1ce0d72513ad service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:57.941127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1165/2309] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:58.435256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:58.436930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:58.451691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9136be-a708-4444-a186-3b498ced9cbe None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:58.458326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:58.458985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:58.459290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:58.459678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] PATH_INFO: `/v3/projects/38335bc4e521473d862606e156feacf3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:58.461522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38335bc4e521473d862606e156feacf3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:58.463478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:58.597996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38335bc4e521473d862606e156feacf3', 'request_id': 'req-8dede119-ee55-4e6d-8dbf-2415864ecc07'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:58.607885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38335bc4e521473d862606e156feacf3', 'request_id': 'req-8dede119-ee55-4e6d-8dbf-2415864ecc07'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:58.620479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38335bc4e521473d862606e156feacf3', 'request_id': 'req-8dede119-ee55-4e6d-8dbf-2415864ecc07'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:58.686322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] The token cache is being invalidate because project 38335bc4e521473d862606e156feacf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:15:58.686639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:58.687272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8dede119-ee55-4e6d-8dbf-2415864ecc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:15:58.688862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1145/2310] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:15:58 2022] DELETE /identity/v3/projects/38335bc4e521473d862606e156feacf3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:58.693685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6750bcc-b6b9-48a3-828a-ed84337c8cdf None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:58.694152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6750bcc-b6b9-48a3-828a-ed84337c8cdf None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:58.694246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6750bcc-b6b9-48a3-828a-ed84337c8cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:58.715106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d6750bcc-b6b9-48a3-828a-ed84337c8cdf None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:58.718676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d6750bcc-b6b9-48a3-828a-ed84337c8cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:58.772274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:58.794947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1166/2311] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:58.796871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2515c5-be85-46f6-b202-44ed183e43a7 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:58.797342 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2515c5-be85-46f6-b202-44ed183e43a7 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:58.797451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2515c5-be85-46f6-b202-44ed183e43a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:58.822016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-cf2515c5-be85-46f6-b202-44ed183e43a7 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:58.823409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cf2515c5-be85-46f6-b202-44ed183e43a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:58.831587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1146/2312] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:58.835952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6750bcc-b6b9-48a3-828a-ed84337c8cdf None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:58.845867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6750bcc-b6b9-48a3-828a-ed84337c8cdf None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:58.850981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:58.851605 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:58.851940 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:58.852219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:58.852952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:15:58.853259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:58.854863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:58.868960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1167/2313] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:15:58.875118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2515c5-be85-46f6-b202-44ed183e43a7 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:58.880895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:58.890088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2515c5-be85-46f6-b202-44ed183e43a7 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:58.892301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:58.892964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:58.893480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:58.893984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:58.894732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:58.896684 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:58.983793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1147/2314] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:15:58 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:58.988128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:58.999431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821a148-e070-41d5-96e4-fd8db22e7515 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:59.058357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:59.059079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:59.059436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:59.059889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:59.060673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:59.062658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:59.161835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1168/2315] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:15:58 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:15:59.168136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:59.179748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18062894-9a8c-4117-9c89-69869d2c5d76 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:59.185378 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:59.185998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:59.186325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:59.186599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:59.187297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:59.188967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:59.208782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1148/2316] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:59.210473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:59.222367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42067e86-f526-4457-9e11-0669682307a8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:59.226986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76a2b4-805b-4177-879e-e27288e6d881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:59.227601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76a2b4-805b-4177-879e-e27288e6d881 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:59.227883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76a2b4-805b-4177-879e-e27288e6d881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:59.228163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76a2b4-805b-4177-879e-e27288e6d881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:59.228894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76a2b4-805b-4177-879e-e27288e6d881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:59.230480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76a2b4-805b-4177-879e-e27288e6d881 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:59.241214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1169/2317] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:15:59.245909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:59.256479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65000be5-2a60-4ac3-bbcc-3c60e3925adc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:59.261712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:59.266017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:59.266109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:59.266404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] PATH_INFO: `/v3/projects/5bbaea344234438dacb64d57891a3dd1/users/eb832931429645a9a0c450875be21c5a/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:59.283044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbaea344234438dacb64d57891a3dd1, user_id=eb832931429645a9a0c450875be21c5a, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:59.285752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:59.350121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1149/2318] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:15:59 2022] PUT /identity/v3/projects/5bbaea344234438dacb64d57891a3dd1/users/eb832931429645a9a0c450875be21c5a/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:15:59.363558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80754d95-871b-4cf7-b2f0-cb55b7da548e None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:59.363849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80754d95-871b-4cf7-b2f0-cb55b7da548e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:59.364129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80754d95-871b-4cf7-b2f0-cb55b7da548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:59.399700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-80754d95-871b-4cf7-b2f0-cb55b7da548e None None] MFA Rules not processed for user `eb832931429645a9a0c450875be21c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:15:59.403182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-80754d95-871b-4cf7-b2f0-cb55b7da548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:59.490249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1170/2319] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:15:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:59.501865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:59.503862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:59.625259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc48b40-623f-45aa-b88b-eac0ae204e08 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:59.629264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7e18f-b615-435b-af97-ece5f6d56fe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:59.629889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7e18f-b615-435b-af97-ece5f6d56fe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:59.630174 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7e18f-b615-435b-af97-ece5f6d56fe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:59.630450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7e18f-b615-435b-af97-ece5f6d56fe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:59.639779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7e18f-b615-435b-af97-ece5f6d56fe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:59.641388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7e18f-b615-435b-af97-ece5f6d56fe4 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:59.656367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1150/2320] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:03.209340 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:03.210010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:03.210010 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:03.229410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] MFA Rules not processed for user `77b0df6c83a84dbcb0fa5802c93dc6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:03.230656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:03.300172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:03.313600 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1171/2321] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:06.616482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22b273-2acd-4a70-81eb-4ac9afdeecd2 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:06.616950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22b273-2acd-4a70-81eb-4ac9afdeecd2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:06.617291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22b273-2acd-4a70-81eb-4ac9afdeecd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:06.637380 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ba22b273-2acd-4a70-81eb-4ac9afdeecd2 None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:06.638572 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ba22b273-2acd-4a70-81eb-4ac9afdeecd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:06.696624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1151/2322] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:16:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:06.738945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:06.740334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:06.819132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5118c-4409-47ae-a0ac-c1176f8278a3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:06.824483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:06.825122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:06.825407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:06.825714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:06.841805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:06.841805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:06.844566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1172/2323] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:16:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:16:07.694599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f5e34-9ac3-48cb-bd73-3ff75fc2d9c2 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:07.695073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f5e34-9ac3-48cb-bd73-3ff75fc2d9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:07.695377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f5e34-9ac3-48cb-bd73-3ff75fc2d9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:07.715190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-858f5e34-9ac3-48cb-bd73-3ff75fc2d9c2 None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:07.716403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-858f5e34-9ac3-48cb-bd73-3ff75fc2d9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:07.723543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1152/2324] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 27 20:16:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:07.736651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:07.747254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5cfc-66a8-4ca3-99ca-e1c34c04212f service swift] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:07.752562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:07.753428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:07.753428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:07.753710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:07.762009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:07.763602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:07.771681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1173/2325] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:16:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:16:08.410295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a9628-eacd-4331-9206-882f19b20b5e None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:08.410586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a9628-eacd-4331-9206-882f19b20b5e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:08.410893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a9628-eacd-4331-9206-882f19b20b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:08.430317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-152a9628-eacd-4331-9206-882f19b20b5e None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:08.431465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-152a9628-eacd-4331-9206-882f19b20b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:08.438557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1153/2326] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:16:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:08.450921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:08.461601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074534a4-4450-482f-93fd-afde916d0382 service swift] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:08.466825 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:08.467443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:08.467729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:08.468000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:08.476158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:08.477797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:08.486094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1174/2327] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:16:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:16:09.116463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:09.116893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:09.117030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:09.139008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:09.140281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:09.148056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1154/2328] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 27 20:16:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:09.161569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:09.172915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129a87f-1c61-43a7-b702-5d71153ab66d service swift] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:09.177968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a672d-c3f4-49e4-86d0-2ac170f5bdb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:09.178584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a672d-c3f4-49e4-86d0-2ac170f5bdb8 service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:09.178946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a672d-c3f4-49e4-86d0-2ac170f5bdb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:09.179231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a672d-c3f4-49e4-86d0-2ac170f5bdb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:09.188669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a672d-c3f4-49e4-86d0-2ac170f5bdb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:09.189634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a672d-c3f4-49e4-86d0-2ac170f5bdb8 service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:09.198651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1175/2329] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:16:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:16:19.131246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=83,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:19.195430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=51 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:21.544446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:21.545862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:21.616697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e40aa-2fb8-48f7-8120-3b488bc208fa None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:21.622021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:21.622675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:21.623043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:21.623325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] PATH_INFO: `/v3/users/412828a7f19c451a9784f370f84c37ec` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:21.624553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=412828a7f19c451a9784f370f84c37ec)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:21.626479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:21.860489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '412828a7f19c451a9784f370f84c37ec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:21.861552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '412828a7f19c451a9784f370f84c37ec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:21.948467 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '412828a7f19c451a9784f370f84c37ec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:21.964164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '412828a7f19c451a9784f370f84c37ec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:21.972479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '412828a7f19c451a9784f370f84c37ec'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:21.979460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1155/2330] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:16:21 2022] DELETE /identity/v3/users/412828a7f19c451a9784f370f84c37ec => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:21.989073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:21.990029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:21.990615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:22.012549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:22.014100 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:22.099331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1176/2331] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:16:22.106814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:22.108407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:22.166123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79881b52-b596-4326-af83-80bea09e3daa admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:22.169857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1b6af-89d1-4ab8-b1d1-dbef70c4007f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:22.170485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1b6af-89d1-4ab8-b1d1-dbef70c4007f service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:22.170759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1b6af-89d1-4ab8-b1d1-dbef70c4007f service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:22.171055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1b6af-89d1-4ab8-b1d1-dbef70c4007f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:22.181507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1b6af-89d1-4ab8-b1d1-dbef70c4007f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:22.182944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1b6af-89d1-4ab8-b1d1-dbef70c4007f service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:22.192769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1156/2332] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:22.414260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:22.416270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:22.442376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109f3b6-8b24-4b9d-857e-39bb2d904021 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:22.449828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:22.450625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:22.450991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:22.451232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] PATH_INFO: `/v3/projects/370ac8b4f98e45f5940e643b3afa6be3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:22.453545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=370ac8b4f98e45f5940e643b3afa6be3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:22.455679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:22.552215 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370ac8b4f98e45f5940e643b3afa6be3', 'request_id': 'req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:22.559111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370ac8b4f98e45f5940e643b3afa6be3', 'request_id': 'req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:22.565007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370ac8b4f98e45f5940e643b3afa6be3', 'request_id': 'req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:22.661984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] The token cache is being invalidate because project 370ac8b4f98e45f5940e643b3afa6be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:16:22.662275 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:22.662883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-aded6d5d-03c9-4990-bbb3-fd7b1f3e5019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:22.664710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1177/2333] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:16:22 2022] DELETE /identity/v3/projects/370ac8b4f98e45f5940e643b3afa6be3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:22.669854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba35a4-c6d4-4003-aea0-ac86f7b7ee79 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:22.670332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba35a4-c6d4-4003-aea0-ac86f7b7ee79 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:22.670427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba35a4-c6d4-4003-aea0-ac86f7b7ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:22.689981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-12ba35a4-c6d4-4003-aea0-ac86f7b7ee79 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:22.691175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-12ba35a4-c6d4-4003-aea0-ac86f7b7ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:22.708561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:22.749610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1157/2334] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:22.817159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b23d63-3e3f-4b41-8fac-2e57711eb181 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:22.817373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b23d63-3e3f-4b41-8fac-2e57711eb181 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:22.817632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b23d63-3e3f-4b41-8fac-2e57711eb181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:22.837632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-44b23d63-3e3f-4b41-8fac-2e57711eb181 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:22.838931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-44b23d63-3e3f-4b41-8fac-2e57711eb181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:22.846892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1178/2335] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:22.851020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba35a4-c6d4-4003-aea0-ac86f7b7ee79 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:22.861582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba35a4-c6d4-4003-aea0-ac86f7b7ee79 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:22.866539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:22.867191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:22.868643 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:22.870464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:22.870464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:16:22.870464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:22.872075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:22.883688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1158/2336] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:16:22.887875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b23d63-3e3f-4b41-8fac-2e57711eb181 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:22.897187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b23d63-3e3f-4b41-8fac-2e57711eb181 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:22.901938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:22.902554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:22.902835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:22.903110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:22.903817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:22.905813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:23.019020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1179/2337] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:16:22 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:16:23.021158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:23.031494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c51f7-fa8d-450b-a7c6-8fd459146fe1 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:23.114653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:23.115366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:23.115724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:23.116178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:23.116968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:23.119820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:23.220867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1159/2338] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:16:23 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:16:23.222231 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:23.233266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f7512-cb30-44d6-9551-60bbf71b9667 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:23.237867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:23.238486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:23.238767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:23.239042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:23.239900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:23.241451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:23.254472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1180/2339] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:16:23.258738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:23.269895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa1681-17f0-4232-b2fd-cfbab4f7fede admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:23.275300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9f4bf-1902-4ed7-bba9-29d4ef8e2e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:23.275920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9f4bf-1902-4ed7-bba9-29d4ef8e2e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:23.276205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9f4bf-1902-4ed7-bba9-29d4ef8e2e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:23.276527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9f4bf-1902-4ed7-bba9-29d4ef8e2e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:23.277231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9f4bf-1902-4ed7-bba9-29d4ef8e2e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:23.278891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9f4bf-1902-4ed7-bba9-29d4ef8e2e5b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:23.303759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1160/2340] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:16:23.307842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:23.318673 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4925c95-df99-457b-825e-a6d1aa6666f0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:23.331914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:23.331914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:23.331914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:23.331914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] PATH_INFO: `/v3/projects/55c519d693b2424b9e0acc02143c6e6a/users/a726227067d2499e9934a824fc4c31a5/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:23.366668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c519d693b2424b9e0acc02143c6e6a, user_id=a726227067d2499e9934a824fc4c31a5, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:23.369625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:23.418976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1181/2341] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:16:23 2022] PUT /identity/v3/projects/55c519d693b2424b9e0acc02143c6e6a/users/a726227067d2499e9934a824fc4c31a5/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:16:23.425217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac6a62-a6b6-4a01-b1ef-bf392ce1eb39 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:23.425506 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac6a62-a6b6-4a01-b1ef-bf392ce1eb39 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:23.425788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac6a62-a6b6-4a01-b1ef-bf392ce1eb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:23.446968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c6ac6a62-a6b6-4a01-b1ef-bf392ce1eb39 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:23.448211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c6ac6a62-a6b6-4a01-b1ef-bf392ce1eb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:23.509069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1161/2342] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:23.519141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:23.520947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:23.542896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:23.579143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01179f4f-e0ff-4b95-bdbc-9078d4c3ff5b admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:23.582875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:23.583448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:23.583733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:23.584007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:23.593902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:23.595438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:23.605446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1182/2343] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:29.599169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee90c2-8159-477c-8eda-fc23c377dc79 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:29.599702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee90c2-8159-477c-8eda-fc23c377dc79 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:29.599702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee90c2-8159-477c-8eda-fc23c377dc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:29.633237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-fdee90c2-8159-477c-8eda-fc23c377dc79 None None] MFA Rules not processed for user `a726227067d2499e9934a824fc4c31a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:29.634600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-fdee90c2-8159-477c-8eda-fc23c377dc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:29.655559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:29.726421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1162/2344] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:29.731191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:29.740957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a3f-cf4d-435c-8297-bac98d42a75f service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:29.751507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:29.751507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:29.751507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:29.751507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:29.752073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:29.754694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:29.867808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1183/2345] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:16:29 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:16:29.872570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee90c2-8159-477c-8eda-fc23c377dc79 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:29.896256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee90c2-8159-477c-8eda-fc23c377dc79 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:29.972080 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:29.973008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:29.973287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:29.973559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:29.974237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:29.976639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:30.066791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1163/2346] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:16:29 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:16:30.071162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:30.082087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ece7c-58f4-4d5e-a9aa-561094adb398 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:30.089061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:30.089697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:30.089980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:30.090283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:30.090987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:30.092745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:30.104759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1184/2347] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:16:30.109086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:30.121396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c26590-9533-4e77-bdd1-97441a9c0b44 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:30.126461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:30.127166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:30.127445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:30.127720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] PATH_INFO: `/v3/projects/a0daac9586ca4509b61961a283414185/users/37f67f2dfc3149ec959198494f2c8169/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:30.144710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0daac9586ca4509b61961a283414185, user_id=37f67f2dfc3149ec959198494f2c8169, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:30.147409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:30.247744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1164/2348] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:16:30 2022] PUT /identity/v3/projects/a0daac9586ca4509b61961a283414185/users/37f67f2dfc3149ec959198494f2c8169/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:30.251734 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:30.262699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93519a2e-cf4d-4b52-ac65-0c87d1683288 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:30.328866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:30.329590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:30.329944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:30.330225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:30.330984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:30.332552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:30.351522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1185/2349] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:16:30.356330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:30.368886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd503f2-a96d-4d76-8a84-3316b09d5a0f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:30.374597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0f4a-aab0-462d-b65e-ea0c651a35e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:30.375680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0f4a-aab0-462d-b65e-ea0c651a35e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:30.375980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0f4a-aab0-462d-b65e-ea0c651a35e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:30.376488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0f4a-aab0-462d-b65e-ea0c651a35e9 admin admin] PATH_INFO: `/v3/projects/a0daac9586ca4509b61961a283414185/users/37f67f2dfc3149ec959198494f2c8169/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:30.378959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0f4a-aab0-462d-b65e-ea0c651a35e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0daac9586ca4509b61961a283414185, user_id=37f67f2dfc3149ec959198494f2c8169, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:30.383823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0f4a-aab0-462d-b65e-ea0c651a35e9 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:30.395415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1165/2350] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:16:30 2022] PUT /identity/v3/projects/a0daac9586ca4509b61961a283414185/users/37f67f2dfc3149ec959198494f2c8169/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:36.547165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:36.549842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:36.666905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4bea4-2053-4a65-b7f5-7184d76dcd30 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:36.699459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:36.736850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:36.737479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:36.737760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:36.738035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:36.739835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:36.752196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:36.753827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:36.764905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1186/2351] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:37.262932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26552d78-1d59-4b42-822e-04a71367581e None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:37.263450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26552d78-1d59-4b42-822e-04a71367581e None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:37.263666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26552d78-1d59-4b42-822e-04a71367581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:37.296587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-26552d78-1d59-4b42-822e-04a71367581e None None] MFA Rules not processed for user `37f67f2dfc3149ec959198494f2c8169`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:37.298202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-26552d78-1d59-4b42-822e-04a71367581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:37.371466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:37.386315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1166/2352] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:37.401522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:37.402956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:37.488341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570733e-ba49-4227-bdbc-1e45e056cf5a service placement] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:37.497303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722e3f3-34cf-4dd4-ab4e-7140a4a3df9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:37.497478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722e3f3-34cf-4dd4-ab4e-7140a4a3df9a service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:37.497749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722e3f3-34cf-4dd4-ab4e-7140a4a3df9a service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:37.498033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722e3f3-34cf-4dd4-ab4e-7140a4a3df9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:37.593327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722e3f3-34cf-4dd4-ab4e-7140a4a3df9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:37.595765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722e3f3-34cf-4dd4-ab4e-7140a4a3df9a service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:37.624606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1187/2353] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:40.110052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26552d78-1d59-4b42-822e-04a71367581e None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:40.111521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-26552d78-1d59-4b42-822e-04a71367581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:40.180042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26552d78-1d59-4b42-822e-04a71367581e None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:40.185587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:40.186396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:40.186522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:40.187010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] PATH_INFO: `/v3/users/58c183e928a1454a97143d4292ae2cf3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:40.192297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c183e928a1454a97143d4292ae2cf3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:40.192297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:40.444761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c183e928a1454a97143d4292ae2cf3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:40.444761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c183e928a1454a97143d4292ae2cf3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:40.446722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07373-6638-41e7-a436-862efc24ed04 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:40.447172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07373-6638-41e7-a436-862efc24ed04 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:40.447499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07373-6638-41e7-a436-862efc24ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:40.466687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f6a07373-6638-41e7-a436-862efc24ed04 None None] MFA Rules not processed for user `a726227067d2499e9934a824fc4c31a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:40.469995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f6a07373-6638-41e7-a436-862efc24ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:40.474729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1188/2354] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:16:40.513089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c183e928a1454a97143d4292ae2cf3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:40.527608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c183e928a1454a97143d4292ae2cf3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:40.534066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c183e928a1454a97143d4292ae2cf3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:40.541253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1167/2355] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:16:40 2022] DELETE /identity/v3/users/58c183e928a1454a97143d4292ae2cf3 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:40.545795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:40.546092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:40.550508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:40.565998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:40.567193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:40.613642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:40.630776 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1189/2356] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:16:40.640858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:40.642498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:40.713863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2d75a-da1e-4f68-84c9-5ebc98d11104 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:40.717923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9896f30-9211-40e5-8606-f287f569a9ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:40.718713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896f30-9211-40e5-8606-f287f569a9ea service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:40.719274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896f30-9211-40e5-8606-f287f569a9ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:40.719726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896f30-9211-40e5-8606-f287f569a9ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:40.740285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9896f30-9211-40e5-8606-f287f569a9ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:40.742477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9896f30-9211-40e5-8606-f287f569a9ea service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:40.754447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1168/2357] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:41.185154 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:41.187166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:41.203408 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9039e-bdaf-460e-af0e-c3336a61e66c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:41.210218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:41.217561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.217561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.217561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] PATH_INFO: `/v3/projects/07f2f5cb5bcb4342a5f798d470a96cd0` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.217561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f2f5cb5bcb4342a5f798d470a96cd0)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:41.220270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:41.364262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f2f5cb5bcb4342a5f798d470a96cd0', 'request_id': 'req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:41.373910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f2f5cb5bcb4342a5f798d470a96cd0', 'request_id': 'req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:41.381429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f2f5cb5bcb4342a5f798d470a96cd0', 'request_id': 'req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:41.473693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] The token cache is being invalidate because project 07f2f5cb5bcb4342a5f798d470a96cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:16:41.474298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:41.475344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5f3aded9-62df-4b33-9ab3-d2d10bda2e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:41.479603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1190/2358] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:16:41 2022] DELETE /identity/v3/projects/07f2f5cb5bcb4342a5f798d470a96cd0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:41.482059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc70d1-c000-4aed-aa32-c0b224f4251d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.482534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc70d1-c000-4aed-aa32-c0b224f4251d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.482630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc70d1-c000-4aed-aa32-c0b224f4251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.513340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-94dc70d1-c000-4aed-aa32-c0b224f4251d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:41.513340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-94dc70d1-c000-4aed-aa32-c0b224f4251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:41.522087 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:41.573712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1169/2359] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:41.581812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84963fd-c2d9-4627-b6f2-a63b0a322d87 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.582071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84963fd-c2d9-4627-b6f2-a63b0a322d87 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.582338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84963fd-c2d9-4627-b6f2-a63b0a322d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.604246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f84963fd-c2d9-4627-b6f2-a63b0a322d87 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:41.605509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f84963fd-c2d9-4627-b6f2-a63b0a322d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:41.614368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1191/2360] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:41.619354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc70d1-c000-4aed-aa32-c0b224f4251d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:41.629111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc70d1-c000-4aed-aa32-c0b224f4251d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:41.633796 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:41.634416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.634699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.634972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.635797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:16:41.636107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:41.637804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:41.649639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1170/2361] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:16:41.654621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84963fd-c2d9-4627-b6f2-a63b0a322d87 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:41.665464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84963fd-c2d9-4627-b6f2-a63b0a322d87 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:41.670317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:41.674700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.674700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.674700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.674700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:41.674700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:41.756620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1192/2362] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:16:41 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:16:41.761132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:41.777838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24210a-2007-4ccf-856b-4839290ddf59 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:41.837205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:41.842263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.842263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.842263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.842263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:41.842263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:41.926867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1171/2363] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:16:41 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:16:41.931187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:41.942463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9f374-c612-4485-8627-d779b3d9020d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:41.948866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:41.949663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.949663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.950023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.950705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:41.952409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:41.964933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1193/2364] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:16:41.969135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:41.980074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5d940-464a-4537-8bcf-4da4463ba7ea admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:41.984909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b391df-9752-42d2-a3b2-ee3304caf482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:41.985539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b391df-9752-42d2-a3b2-ee3304caf482 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:41.985819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b391df-9752-42d2-a3b2-ee3304caf482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:41.986091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b391df-9752-42d2-a3b2-ee3304caf482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:41.986782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b391df-9752-42d2-a3b2-ee3304caf482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:41.988446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b391df-9752-42d2-a3b2-ee3304caf482 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:41.999344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1172/2365] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:16:42.003417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:42.013427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dc43a-d7fe-4139-b2e5-ff968c0c52e1 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:42.018324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:42.019036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:42.019313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:42.019586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] PATH_INFO: `/v3/projects/90bc4e6b47db4ddc87cbc5822eb65472/users/d474050bcc464edcbc10d2833f2b3b85/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:42.034830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bc4e6b47db4ddc87cbc5822eb65472, user_id=d474050bcc464edcbc10d2833f2b3b85, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:42.037490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:42.109266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1194/2366] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:16:42 2022] PUT /identity/v3/projects/90bc4e6b47db4ddc87cbc5822eb65472/users/d474050bcc464edcbc10d2833f2b3b85/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:42.180064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234d22-512f-4da5-9003-ba37989b6547 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:42.180347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234d22-512f-4da5-9003-ba37989b6547 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:42.180659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4234d22-512f-4da5-9003-ba37989b6547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:42.217004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d4234d22-512f-4da5-9003-ba37989b6547 None None] MFA Rules not processed for user `d474050bcc464edcbc10d2833f2b3b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:42.218272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d4234d22-512f-4da5-9003-ba37989b6547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:42.284290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1173/2367] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:42.295521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:42.297246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:42.357872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804383af-6f28-44c2-925b-41722e406884 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:42.370981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea83a3d-2bd2-4108-a808-2ef2136c7a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:42.371618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea83a3d-2bd2-4108-a808-2ef2136c7a85 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:42.371898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea83a3d-2bd2-4108-a808-2ef2136c7a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:42.372182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea83a3d-2bd2-4108-a808-2ef2136c7a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:42.384843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea83a3d-2bd2-4108-a808-2ef2136c7a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:42.387397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea83a3d-2bd2-4108-a808-2ef2136c7a85 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:42.401244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1195/2368] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:42.426243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234d22-512f-4da5-9003-ba37989b6547 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:42.439528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:42.488420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4234d22-512f-4da5-9003-ba37989b6547 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:42.493271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:42.493912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:42.494196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:42.494474 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] PATH_INFO: `/v3/users/d474050bcc464edcbc10d2833f2b3b85` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:42.495658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d474050bcc464edcbc10d2833f2b3b85)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:42.497613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:42.721851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd474050bcc464edcbc10d2833f2b3b85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:42.722531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd474050bcc464edcbc10d2833f2b3b85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:42.787833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd474050bcc464edcbc10d2833f2b3b85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:42.800765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd474050bcc464edcbc10d2833f2b3b85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:42.808182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd474050bcc464edcbc10d2833f2b3b85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:42.814155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1174/2369] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:16:42 2022] DELETE /identity/v3/users/d474050bcc464edcbc10d2833f2b3b85 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:42.818732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060c268-820f-40c7-9238-8f678079e549 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:42.819194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060c268-820f-40c7-9238-8f678079e549 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:42.819312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060c268-820f-40c7-9238-8f678079e549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:42.839006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e060c268-820f-40c7-9238-8f678079e549 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:42.840200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e060c268-820f-40c7-9238-8f678079e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:42.845149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:42.904985 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1196/2370] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:42.915032 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:42.916582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:42.978861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5648e-5eb7-4683-85c7-cfb56cfd02dc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:42.982639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26f224-b2c7-4c19-94f3-50a38676ed6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:42.983272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26f224-b2c7-4c19-94f3-50a38676ed6c service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:42.983543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26f224-b2c7-4c19-94f3-50a38676ed6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:42.983820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26f224-b2c7-4c19-94f3-50a38676ed6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:42.998381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26f224-b2c7-4c19-94f3-50a38676ed6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:42.999949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26f224-b2c7-4c19-94f3-50a38676ed6c service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:43.011030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1175/2371] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:43.462578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060c268-820f-40c7-9238-8f678079e549 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:43.464016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e060c268-820f-40c7-9238-8f678079e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:43.476652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060c268-820f-40c7-9238-8f678079e549 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:43.482523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:43.483341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:43.483692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:43.489365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] PATH_INFO: `/v3/projects/90bc4e6b47db4ddc87cbc5822eb65472` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:43.489365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90bc4e6b47db4ddc87cbc5822eb65472)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:43.489365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:43.588466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90bc4e6b47db4ddc87cbc5822eb65472', 'request_id': 'req-433b6bc7-5198-40eb-aa07-f0bc72f0e763'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:43.594906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90bc4e6b47db4ddc87cbc5822eb65472', 'request_id': 'req-433b6bc7-5198-40eb-aa07-f0bc72f0e763'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:43.601013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90bc4e6b47db4ddc87cbc5822eb65472', 'request_id': 'req-433b6bc7-5198-40eb-aa07-f0bc72f0e763'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:43.687410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] The token cache is being invalidate because project 90bc4e6b47db4ddc87cbc5822eb65472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:16:43.688501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:43.688501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:43.690121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1197/2372] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:16:43 2022] DELETE /identity/v3/projects/90bc4e6b47db4ddc87cbc5822eb65472 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:49.329431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b91f8-4abc-49d5-ab87-cb77a02f8978 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:49.329881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b91f8-4abc-49d5-ab87-cb77a02f8978 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:49.330051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b91f8-4abc-49d5-ab87-cb77a02f8978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:49.350400 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-397b91f8-4abc-49d5-ab87-cb77a02f8978 None None] MFA Rules not processed for user `8f2d8cdf0e0d48f4a89a0e85694c3372`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:49.352132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-397b91f8-4abc-49d5-ab87-cb77a02f8978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:49.383464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:49.408293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1176/2373] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:53.683152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:16:57.404475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:57.405975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:57.469008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b6bc7-5198-40eb-aa07-f0bc72f0e763 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:57.473562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:57.474202 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:57.474485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:57.474764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] PATH_INFO: `/v3/users/eb832931429645a9a0c450875be21c5a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:57.475961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb832931429645a9a0c450875be21c5a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:57.479314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:57.623949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb832931429645a9a0c450875be21c5a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:57.624565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb832931429645a9a0c450875be21c5a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:57.678759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb832931429645a9a0c450875be21c5a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:57.695519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb832931429645a9a0c450875be21c5a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:57.696778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb832931429645a9a0c450875be21c5a'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:57.709651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1198/2374] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:16:57 2022] DELETE /identity/v3/users/eb832931429645a9a0c450875be21c5a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:16:57.714335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:57.714832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:57.714949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:57.734226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:16:57.735412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:57.801285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1177/2375] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:16:57.811754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:57.813150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:57.872683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159585f-0193-4e2d-aa3b-6b22efb9bbef admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:57.876624 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb799a7-53fd-47a4-abdf-7dd448e53996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:57.877320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb799a7-53fd-47a4-abdf-7dd448e53996 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:57.877603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb799a7-53fd-47a4-abdf-7dd448e53996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:57.877883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb799a7-53fd-47a4-abdf-7dd448e53996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:57.889678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb799a7-53fd-47a4-abdf-7dd448e53996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:57.891136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb799a7-53fd-47a4-abdf-7dd448e53996 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:57.904052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1199/2376] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:58.329115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:16:58.329115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:16:58.336951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f06e7d-33ec-44bd-a0b3-5180f728c7fc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:16:58.341733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:16:58.342381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:16:58.342658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:16:58.342931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] PATH_INFO: `/v3/projects/5bbaea344234438dacb64d57891a3dd1` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:16:58.344101 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bbaea344234438dacb64d57891a3dd1)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:16:58.346013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:16:58.462972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bbaea344234438dacb64d57891a3dd1', 'request_id': 'req-e6fe74dc-4ace-4358-885a-e5c74770f9ee'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:58.469433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bbaea344234438dacb64d57891a3dd1', 'request_id': 'req-e6fe74dc-4ace-4358-885a-e5c74770f9ee'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:58.475065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bbaea344234438dacb64d57891a3dd1', 'request_id': 'req-e6fe74dc-4ace-4358-885a-e5c74770f9ee'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:58.557826 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] The token cache is being invalidate because project 5bbaea344234438dacb64d57891a3dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:16:58.558116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:58.558740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:16:58.560294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1178/2377] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:16:58 2022] DELETE /identity/v3/projects/5bbaea344234438dacb64d57891a3dd1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:17:07.897941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:17:08.554726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:17:19.247452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:17:19.248148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:17:19.248461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:17:19.269004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:17:19.270337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:17:19.336615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1200/2378] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:17:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:17:19.470634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:17:19.471187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:17:19.541139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe74dc-4ace-4358-885a-e5c74770f9ee admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:17:19.546200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:17:19.546813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:17:19.547096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:17:19.547370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:17:19.555681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:17:19.557805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:17:19.565852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1179/2379] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:17:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 20:17:29.328685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=25 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:17:29.562634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=32 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:18:13.861164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:13.862668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:13.903253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:13.905409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:13.928596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f413e0-4ed4-4035-9314-fb524e4bb5cc None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:13.932707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:13.936592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:13.936592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:13.936592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:13.936592 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:13.969703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ede01-d005-42e1-8f07-1abef78e2cfd service swift] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:13.973493 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59f193-e46e-4fdb-ae0d-7ad8e98c13b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:13.974106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f193-e46e-4fdb-ae0d-7ad8e98c13b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:13.974412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f193-e46e-4fdb-ae0d-7ad8e98c13b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:13.974685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59f193-e46e-4fdb-ae0d-7ad8e98c13b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:13.976503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3a59f193-e46e-4fdb-ae0d-7ad8e98c13b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:14.015357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:14.022059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:14.028282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1201/2380] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:18:14.060398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59f193-e46e-4fdb-ae0d-7ad8e98c13b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:14.061875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59f193-e46e-4fdb-ae0d-7ad8e98c13b0 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:14.073933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1180/2381] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:18:23.292218 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:23.300950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65040fae-4ce9-4e8e-a7f3-a3469889f8bf service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:23.304763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547bc2a-81a6-4609-bb5f-04123af06c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:23.305382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547bc2a-81a6-4609-bb5f-04123af06c1e service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:23.305667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547bc2a-81a6-4609-bb5f-04123af06c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:23.305947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547bc2a-81a6-4609-bb5f-04123af06c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:23.307758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d547bc2a-81a6-4609-bb5f-04123af06c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:23.381454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547bc2a-81a6-4609-bb5f-04123af06c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:23.383472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547bc2a-81a6-4609-bb5f-04123af06c1e service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:23.393276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1202/2382] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:18:23.499905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe63db-f312-4bb5-a735-f1c12a1aaca3 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:23.500655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe63db-f312-4bb5-a735-f1c12a1aaca3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:23.501162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe63db-f312-4bb5-a735-f1c12a1aaca3 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:23.502799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1181/2383] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:18:23.510755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6c51-d36c-4772-9d23-2226644ddce3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:23.511145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6c51-d36c-4772-9d23-2226644ddce3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:23.511496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6c51-d36c-4772-9d23-2226644ddce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:23.532843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-15ed6c51-d36c-4772-9d23-2226644ddce3 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:18:23.534155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-15ed6c51-d36c-4772-9d23-2226644ddce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:23.543858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1203/2384] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:18:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:18:23.555517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe63db-f312-4bb5-a735-f1c12a1aaca3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:23.557249 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-61fe63db-f312-4bb5-a735-f1c12a1aaca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:23.618399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe63db-f312-4bb5-a735-f1c12a1aaca3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:23.621784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48a76-5db7-4cf6-ac1d-9cd33f3644ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:23.622402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48a76-5db7-4cf6-ac1d-9cd33f3644ee service glance] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:23.622683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48a76-5db7-4cf6-ac1d-9cd33f3644ee service glance] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:23.622958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48a76-5db7-4cf6-ac1d-9cd33f3644ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:23.634110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48a76-5db7-4cf6-ac1d-9cd33f3644ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:23.635551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48a76-5db7-4cf6-ac1d-9cd33f3644ee service glance] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:23.648968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1182/2385] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:18:23.871739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccfc03-83ef-491d-a11d-09ad226d9318 None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:23.871962 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccfc03-83ef-491d-a11d-09ad226d9318 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:23.872243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccfc03-83ef-491d-a11d-09ad226d9318 None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:23.873753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1204/2386] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:18:23.879595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19064b1-cccc-4bb0-b200-91c3cd996be8 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:23.879902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19064b1-cccc-4bb0-b200-91c3cd996be8 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:23.880179 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19064b1-cccc-4bb0-b200-91c3cd996be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:23.901242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d19064b1-cccc-4bb0-b200-91c3cd996be8 None None] MFA Rules not processed for user `54aedfbb64c54c589003b335fde2a17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:18:23.902500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d19064b1-cccc-4bb0-b200-91c3cd996be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:23.909248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1183/2387] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:18:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:18:23.922647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccfc03-83ef-491d-a11d-09ad226d9318 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:23.924485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0fccfc03-83ef-491d-a11d-09ad226d9318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:23.938623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccfc03-83ef-491d-a11d-09ad226d9318 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:23.942398 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be33f14-e38a-467c-a9c7-651435c243b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:23.943025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be33f14-e38a-467c-a9c7-651435c243b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:23.943305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be33f14-e38a-467c-a9c7-651435c243b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:23.943577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be33f14-e38a-467c-a9c7-651435c243b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:23.952223 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be33f14-e38a-467c-a9c7-651435c243b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:23.953934 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be33f14-e38a-467c-a9c7-651435c243b3 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:23.963006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1205/2388] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:18:24.097713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ee50-1597-4641-b3fe-5922753c354c None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:24.098003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ee50-1597-4641-b3fe-5922753c354c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:24.098280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ee50-1597-4641-b3fe-5922753c354c None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:24.099651 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1184/2389] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:18:24.105587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:24.105874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:24.106153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:24.127460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] MFA Rules not processed for user `784880e276bf4d64ae785609a463403a`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:18:24.128730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:24.138770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1206/2390] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:18:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:18:24.149223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438ee50-1597-4641-b3fe-5922753c354c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:24.162036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438ee50-1597-4641-b3fe-5922753c354c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:24.166418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:24.167580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:24.167871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:24.168153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:24.180195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:24.181192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:24.192505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1185/2391] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:18:26.098513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:26.100096 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:26.113562 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:18:26.173572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a14790-0a10-40c1-a754-8ebef5f382a3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:26.177945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:26.178799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:26.179315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:26.179793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:26.182668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:26.259110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:26.260737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:26.274169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1207/2392] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:18:29.115982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:18:29.125805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c0080-9493-4759-ae57-883b5a3d5538 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:18:29.129413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:18:29.130015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:18:29.130299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:18:29.130574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:18:29.133003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:18:29.154084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:18:29.191042 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:18:29.192499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:18:29.202540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1186/2393] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:18:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:18:36.314974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=44 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:18:39.198534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=17 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:19:29.675760 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:29.685319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da0bd9-f0ae-408c-82b8-a6d3f735f79f service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:29.688956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:29.689572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:29.689864 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:29.690137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:29.691896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:19:29.752792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:29.754239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:29.763460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1208/2394] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:19:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:19:39.759255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=28 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:20:24.368566 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:20:24.379276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9aea-4951-472f-a30e-134985d0a0ac service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:20:24.382942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:20:24.384336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:20:24.384336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:20:24.384336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:20:24.386240 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:20:24.396749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:20:24.398175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:20:24.406531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1187/2395] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:20:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:20:24.788182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:20:24.789639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:20:24.806047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17292e8-c009-4ff7-9dd2-f64c998a93be service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:20:24.811739 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:20:24.812359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:20:24.812671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:20:24.812947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:20:24.814722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:20:24.874732 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:20:24.876342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:20:24.885332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1209/2396] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:20:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:20:34.402775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:20:34.881265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=28 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:20:40.901890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:20:40.916206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc6d6f-9882-442d-a315-f4d838168c46 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:20:40.921700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:20:40.922427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:20:40.922712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:20:40.922992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:20:40.924903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:20:40.938596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:20:40.940244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:20:40.951659 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1188/2397] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:20:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:20:50.947089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:21:37.228418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:21:37.229896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:21:37.319846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1d031-f17e-4d9c-9548-8daeb6c045a9 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:21:37.325414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:21:37.326025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:21:37.326306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:21:37.326583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:21:37.328306 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:21:37.390644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:21:37.393725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:21:37.403130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1210/2398] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:21:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:21:47.399055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=62 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:22:27.825987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:22:27.827379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:22:27.962165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0488b-9a27-42e4-aaa5-a71aabf31cce service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:22:27.967265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:22:27.967881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:22:27.968163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:22:27.968471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:22:27.970739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:22:27.982201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:22:27.984069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:22:27.994600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1189/2399] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:22:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:22:37.990410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=63 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:23:14.057699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:23:14.077565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17281f6f-c5f9-470c-b1eb-6a00b6473012 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:23:14.081236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:23:14.081861 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:23:14.082123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:23:14.082396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:23:14.092470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:23:14.094370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:23:14.104977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1211/2400] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:23:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:23:14.932081 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:23:14.941727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389e1f3-ac03-4a14-8d58-64c4c7ebd396 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:23:14.945307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:23:14.945950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:23:14.946211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:23:14.946483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:23:14.957158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:23:14.958562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:23:14.969778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1190/2401] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:23:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:23:24.100298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=10 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:23:24.964834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:24:17.495344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:24:17.496854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:24:17.530540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a04ae-d31e-4070-9a94-2e2db8191a81 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:24:17.534249 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:24:17.534858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:24:17.536005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:24:17.536005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:24:17.544314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:24:17.545749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:24:17.572271 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1212/2402] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:24:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:24:17.638418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:24:17.647987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055040-933c-4ff6-bfe4-32a305ef5287 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:24:17.652406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:24:17.653728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:24:17.654049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:24:17.654292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:24:17.663863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:24:17.665461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:24:17.694871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1191/2403] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:24:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:24:27.569699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=22 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:24:27.692071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=13 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:24:37.844667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:24:37.853749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730596a-b2b6-4111-9854-9dbddcbaabf3 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:24:37.857901 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:24:37.858581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:24:37.859034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:24:37.859391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:24:37.869356 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:24:37.870773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:24:37.898372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1213/2404] 10.222.0.47 () {60 vars in 1396 bytes} [Wed Jul 27 20:24:37 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:24:47.895527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=13 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:25:29.221971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:29.231551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e25bb-73a1-4450-9b23-4a739c14c50e service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:29.235162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:29.235764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:29.236068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:29.236412 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:29.243492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:29.244975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:29.253593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1192/2405] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:25:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:39.249798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:26:17.979526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:26:17.988753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beff218-ba04-499e-9467-b1b79e952237 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:26:17.992502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:26:17.993234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:26:17.993516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:26:17.993788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:26:18.002688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:26:18.004191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:26:18.031255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1214/2406] 10.222.0.47 () {60 vars in 1396 bytes} [Wed Jul 27 20:26:17 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:26:28.028660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=13 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:26:38.144995 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:26:38.154210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a32442-e267-4353-9b36-ee614921c0c0 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:26:38.157988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:26:38.158598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:26:38.158875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:26:38.159146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:26:38.167902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:26:38.169354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:26:38.196621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1193/2407] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:26:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:26:48.193920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=13 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:27:28.732601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:27:28.734030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:27:28.749198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a9caf-8690-4600-8e92-b02e651eac3f service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:27:28.754109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:27:28.754703 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:27:28.754980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:27:28.755250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:27:28.763537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:27:28.765114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:27:28.776430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1215/2408] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:27:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:27:38.770720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:28:18.122741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:28:18.131655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea9e6c-0b3d-4528-8371-6b592fca1a19 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:28:18.135383 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:28:18.135990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:28:18.136269 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:28:18.136573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:28:18.138409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:28:18.208877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:28:18.210281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:28:18.221794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1194/2409] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:28:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:28:18.265912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:28:18.282484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67be683-adb5-452b-ae97-9198cff5a948 service placement] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:28:18.282484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:28:18.282484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:28:18.282484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:28:18.283321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:28:18.283321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:28:18.297483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:28:18.299073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:28:18.311121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1216/2410] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:28:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:28:18.783532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:28:18.794966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93025-38d5-414c-87ba-a932ee693b8c service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:28:18.800498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:28:18.801143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:28:18.801487 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:28:18.801766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:28:18.803525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:28:18.815548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:28:18.817221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:28:18.828272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1195/2411] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:28:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:28:28.305832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:28:28.824423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=40 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:30:18.441550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:18.503171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2dfda-be1c-40f0-8f0f-6cfcc5930784 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:18.507349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:18.507977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:18.508256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:18.508565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:18.510518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:30:18.587190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:18.588806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:18.598681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1217/2412] 10.222.0.47 () {60 vars in 1396 bytes} [Wed Jul 27 20:30:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:30:18.849354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:18.860760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb89935-83eb-437b-8f40-9cf69820c74e service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:18.866531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:18.867127 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:18.867406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:18.867681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:18.869558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:30:18.945115 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:18.946655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:18.956873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1196/2413] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:30:19.122805 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:19.135872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11c65f-441f-40c4-8149-5b72fae2d425 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:19.141933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:19.142539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:19.142817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:19.143088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:19.144919 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:30:19.217153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:19.218740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:19.227742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1218/2414] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:30:28.952809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=33 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:30:29.223526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=86 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:30:40.666902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:40.680727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f42a5-4443-4e16-b392-660727b3826d service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:40.688932 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:40.689676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:40.690057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:40.690433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:40.692479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:30:40.707259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:40.712671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:40.737989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1197/2415] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:30:40.878240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:40.881143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:30:41.044002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5c8f7-49cb-419b-b63a-9e42c38f1f29 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:41.049203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:41.050045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:41.050507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:41.050917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:41.052893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:30:41.090733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:41.092456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:41.109155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1219/2416] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:30:50.718092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:30:51.099004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=31 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:19.051340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:19.052967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:19.066893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8442fa-b669-47b9-afad-f9e2461944a8 service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:19.077630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:19.077630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:19.077630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:19.077630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:19.078015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:19.151931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:19.153551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:19.168102 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1198/2417] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:31:29.159122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=33 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:39.056005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:39.084984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12536e21-6c47-4fb7-84cc-1475c0281e7d service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:39.090378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:39.090983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:39.091261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:39.091534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:39.093374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:39.161801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:39.163371 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:39.172346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1220/2418] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:31:49.168092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=40 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:32:35.603089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:32:35.714352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fcda8-8ed9-40b6-90de-48ffea8248d5 service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:32:35.719445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:32:35.720048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:32:35.720360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:32:35.720667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:32:35.722529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:32:35.755063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:32:35.756641 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:32:35.766205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1199/2419] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:32:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:32:45.762100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=63 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:33:18.194806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:33:18.223695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb58e0f-38a6-4fd5-8b32-87d8df06d831 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:33:18.227284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:33:18.227882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:33:18.228156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:33:18.228450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:33:18.238458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:33:18.239916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:33:18.251062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1221/2420] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:33:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:33:19.357298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:33:19.367152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd036b-9594-49da-a5a2-1e835f137ee3 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:33:19.371070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:33:19.371678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:33:19.371961 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:33:19.372236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:33:19.382371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:33:19.383791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:33:19.394602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1200/2421] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:33:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:33:28.246317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=15 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:33:29.389817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:20.177456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:20.179093 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:20.214543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86814418-13bb-430d-8579-f8e483991cc3 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:20.218362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:20.218978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:20.219255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:20.219526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:20.229486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:20.230902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:20.257203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1222/2422] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:20.304327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:20.313606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e835c7-4d4f-4d07-ad24-63dca389b580 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:20.317296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:20.317861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:20.318139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:20.318411 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:20.326658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:20.328068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:20.355921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1201/2423] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:20.404727 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:20.414212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea4d62-7504-422f-8299-c3ae75f9bb42 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:20.417904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:20.418514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:20.418801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:20.419078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:20.427795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:20.429242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:20.457128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1223/2424] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:20.500051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:20.509066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470e4f4-7492-4787-af2a-4ce421e306b4 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:20.512881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:20.513484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:20.513765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:20.514041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:20.522959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:20.524456 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:20.550858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1202/2425] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:30.453895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=35 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:30.548194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=26 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:37.642922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:37.655255 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cd2b3-9493-4b14-8795-8b546a4fd858 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:37.661237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:37.661836 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:37.662116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:37.662392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:37.670000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:37.671570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:37.680900 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1224/2426] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:40.465176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:40.484437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534ae5d-71b4-4a53-8c46-88ba0aa9d83b service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:40.489329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:40.490255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:40.490715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:40.491260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:40.493848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:40.600927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:40.602338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:40.621426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1203/2427] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:41.574835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:41.585037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9f33-e0cf-4b0b-b393-51135beea1f1 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:41.588785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:41.589431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:41.589715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:41.589990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:41.597633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:41.599073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:41.607044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1225/2428] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:44.651177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:44.654436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:44.669609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2da69-8742-4a48-a91f-7780743af485 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:44.677964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:44.678715 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:44.679242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:44.679744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:44.682418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:44.709325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:44.710735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:44.721650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1204/2429] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:48.707266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:48.709014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:48.724254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea669-132e-4a9f-8e26-5c891e251e4f service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:48.728963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:48.729593 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:48.729876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:48.730153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] PATH_INFO: `/v3/users/a726227067d2499e9934a824fc4c31a5` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:48.731325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a726227067d2499e9934a824fc4c31a5)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:48.733319 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:48.907984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a726227067d2499e9934a824fc4c31a5'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:48.908652 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a726227067d2499e9934a824fc4c31a5'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:48.980451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a726227067d2499e9934a824fc4c31a5'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:48.993557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a726227067d2499e9934a824fc4c31a5'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:48.994262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a726227067d2499e9934a824fc4c31a5'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:49.008039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1226/2430] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:36:48 2022] DELETE /identity/v3/users/a726227067d2499e9934a824fc4c31a5 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:36:49.267653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:49.280767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a723d5-4dd8-4b9d-9aeb-e735848cdb89 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:49.339071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:49.339719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:49.340025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:49.340301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] PATH_INFO: `/v3/projects/55c519d693b2424b9e0acc02143c6e6a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:49.351229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c519d693b2424b9e0acc02143c6e6a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:49.353195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:49.457940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:49.460762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c519d693b2424b9e0acc02143c6e6a', 'request_id': 'req-9b85f028-3aa5-4749-9764-70e2a9c8872d'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:49.467563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c519d693b2424b9e0acc02143c6e6a', 'request_id': 'req-9b85f028-3aa5-4749-9764-70e2a9c8872d'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:49.473536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c519d693b2424b9e0acc02143c6e6a', 'request_id': 'req-9b85f028-3aa5-4749-9764-70e2a9c8872d'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:49.648451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] The token cache is being invalidate because project 55c519d693b2424b9e0acc02143c6e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:36:49.648744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:49.649377 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:49.650902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1205/2431] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:36:49 2022] DELETE /identity/v3/projects/55c519d693b2424b9e0acc02143c6e6a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:36:49.655031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:49.656484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:49.669965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a916-2f54-4294-b6e4-0db454cf56b6 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:49.728372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:49.729146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:49.729428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:49.729707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] PATH_INFO: `/v3/users/37f67f2dfc3149ec959198494f2c8169` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:49.743893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f67f2dfc3149ec959198494f2c8169)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:49.746387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:49.917638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:49.926332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f67f2dfc3149ec959198494f2c8169'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:49.927030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f67f2dfc3149ec959198494f2c8169'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.022803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f67f2dfc3149ec959198494f2c8169'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.036292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f67f2dfc3149ec959198494f2c8169'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.037013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b0bb4d08-5ae7-45df-991a-613568e93063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f67f2dfc3149ec959198494f2c8169'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.049183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1227/2432] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:36:49 2022] DELETE /identity/v3/users/37f67f2dfc3149ec959198494f2c8169 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:36:50.295862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:50.297458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:50.311390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85f028-3aa5-4749-9764-70e2a9c8872d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:50.372294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:50.372892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:50.373170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:50.373447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] PATH_INFO: `/v3/projects/a0daac9586ca4509b61961a283414185` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:50.384323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0daac9586ca4509b61961a283414185)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:50.386307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:50.517247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0daac9586ca4509b61961a283414185', 'request_id': 'req-8f148312-3266-4247-a1eb-6fc95b083424'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.523794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0daac9586ca4509b61961a283414185', 'request_id': 'req-8f148312-3266-4247-a1eb-6fc95b083424'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.529413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0daac9586ca4509b61961a283414185', 'request_id': 'req-8f148312-3266-4247-a1eb-6fc95b083424'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.602728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] The token cache is being invalidate because project a0daac9586ca4509b61961a283414185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:36:50.603020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.603738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8f148312-3266-4247-a1eb-6fc95b083424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:36:50.605333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1206/2433] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:36:50 2022] DELETE /identity/v3/projects/a0daac9586ca4509b61961a283414185 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:36:50.610293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33f01-e40b-411a-8796-b68f3a8b1a17 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:50.610581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33f01-e40b-411a-8796-b68f3a8b1a17 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:50.610857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df33f01-e40b-411a-8796-b68f3a8b1a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:50.667807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-4df33f01-e40b-411a-8796-b68f3a8b1a17 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:36:50.668955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4df33f01-e40b-411a-8796-b68f3a8b1a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:50.726745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1228/2434] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:36:50.734568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde9983-03d6-44df-93b1-907389480e99 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:50.734855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde9983-03d6-44df-93b1-907389480e99 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:50.735132 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde9983-03d6-44df-93b1-907389480e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:50.754972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1bde9983-03d6-44df-93b1-907389480e99 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:36:50.756170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1bde9983-03d6-44df-93b1-907389480e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:50.764338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1207/2435] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:36:50.768622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33f01-e40b-411a-8796-b68f3a8b1a17 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:50.778938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df33f01-e40b-411a-8796-b68f3a8b1a17 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:50.783475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:50.784087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:50.784367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:50.784676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:50.786035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:36:50.786339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:50.787929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:50.799308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1229/2436] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:36:50.803521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde9983-03d6-44df-93b1-907389480e99 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:50.812883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde9983-03d6-44df-93b1-907389480e99 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:50.817708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:50.818308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:50.818589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:50.818864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:50.819530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:50.821460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:50.920073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1208/2437] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:36:50 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:36:50.924040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:50.934086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8249fd-444f-4b73-a678-36e306819756 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:50.994555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:50.995179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:50.995463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:50.995741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:50.996862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:50.998880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:51.072971 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:51.078257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1230/2438] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:36:50 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:36:51.084984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:51.092884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb5d00-9287-49dd-bb3c-644cf9feb1e2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:51.098330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:51.099038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:51.099319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:51.099597 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:51.100298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:51.101935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:51.111814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1209/2439] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:36:51.116230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:51.128294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a95f4-3680-4562-829d-ce0bcead85a3 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:51.133644 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307d713-d708-4a98-aea2-3736b321e755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:51.134682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307d713-d708-4a98-aea2-3736b321e755 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:51.134682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307d713-d708-4a98-aea2-3736b321e755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:51.134883 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307d713-d708-4a98-aea2-3736b321e755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:51.135503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307d713-d708-4a98-aea2-3736b321e755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:51.137923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307d713-d708-4a98-aea2-3736b321e755 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:51.150399 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1231/2440] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:36:51.154352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:51.164883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9f2-141c-4534-a9bf-ee79ab4bae7d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:51.169517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:51.170223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:51.170451 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:51.170730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] PATH_INFO: `/v3/projects/f9befc2c90574dae9c2f9d087b909062/users/3864d76d4c3b478bad9e5c3c7024a062/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:51.172986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:51.186565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9befc2c90574dae9c2f9d087b909062, user_id=3864d76d4c3b478bad9e5c3c7024a062, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:51.191768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:51.274835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1210/2441] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:36:51 2022] PUT /identity/v3/projects/f9befc2c90574dae9c2f9d087b909062/users/3864d76d4c3b478bad9e5c3c7024a062/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:36:51.279925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929285a7-7562-4d98-949b-ae2961ee673c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:51.280502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929285a7-7562-4d98-949b-ae2961ee673c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:51.285875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929285a7-7562-4d98-949b-ae2961ee673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:51.300267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-929285a7-7562-4d98-949b-ae2961ee673c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:36:51.302130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-929285a7-7562-4d98-949b-ae2961ee673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:51.361746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1232/2442] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:36:51.372251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:51.374033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:51.433746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f572a8-14f5-4bf0-9980-34221bbb5815 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:51.437538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:51.438154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:51.438679 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:51.438953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:51.447966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:51.449426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:51.458852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1211/2443] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:56.526461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d5714-ac74-4338-b118-4e1474c70caa None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:56.526975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d5714-ac74-4338-b118-4e1474c70caa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:56.526975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d5714-ac74-4338-b118-4e1474c70caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:56.558854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8d0d5714-ac74-4338-b118-4e1474c70caa None None] MFA Rules not processed for user `3864d76d4c3b478bad9e5c3c7024a062`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:36:56.560080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8d0d5714-ac74-4338-b118-4e1474c70caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:56.645785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1233/2444] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:36:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:36:56.646639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:56.657089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a97749-9a27-4c9e-b5c5-18d61c591659 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:56.662286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:56.662913 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:56.663197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:56.663660 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:56.664501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:56.666420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:56.765709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1212/2445] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:36:56 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:36:56.770024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d5714-ac74-4338-b118-4e1474c70caa None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:56.781474 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d5714-ac74-4338-b118-4e1474c70caa None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:56.849663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:56.850553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:56.850811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:56.851111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:56.851796 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:56.853917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:57.017606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1234/2446] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:36:56 2022] POST /identity/v3/users => generated 296 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:36:57.022415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:57.033375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf15f9e-ce92-4ebd-ace7-7ef04f5f434c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:57.038482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:57.039118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:57.039398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:57.040004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:57.040914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:57.042722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:57.058630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1213/2447] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:36:57.059849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:57.073966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351523e-61c3-410c-9b4b-ceff508295f8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:57.078977 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:57.079651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:57.079933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:57.080212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] PATH_INFO: `/v3/projects/ab13f7c8ebd44e639e4779c9647bd90d/users/3bf7d765887948e184efbe75e222eb43/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:57.083545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:57.095859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab13f7c8ebd44e639e4779c9647bd90d, user_id=3bf7d765887948e184efbe75e222eb43, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:57.098597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:57.175026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1235/2448] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:36:57 2022] PUT /identity/v3/projects/ab13f7c8ebd44e639e4779c9647bd90d/users/3bf7d765887948e184efbe75e222eb43/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:36:57.179620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:57.189775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a4376-31f6-41cb-b956-f066c2ee1cfe admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:57.246620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1dd17-98b3-471f-ab2d-2ec5b984d4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:57.247244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1dd17-98b3-471f-ab2d-2ec5b984d4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:57.247531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1dd17-98b3-471f-ab2d-2ec5b984d4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:57.247811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1dd17-98b3-471f-ab2d-2ec5b984d4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:57.248637 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1dd17-98b3-471f-ab2d-2ec5b984d4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:57.250246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1dd17-98b3-471f-ab2d-2ec5b984d4fb admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:57.260086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1214/2449] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:36:57.264328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:57.274689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd283bd-5211-47af-ae2d-2d3ea1cf68ba admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:57.279278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:57.288694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:57.288694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:57.288694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] PATH_INFO: `/v3/projects/ab13f7c8ebd44e639e4779c9647bd90d/users/3bf7d765887948e184efbe75e222eb43/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:57.288694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab13f7c8ebd44e639e4779c9647bd90d, user_id=3bf7d765887948e184efbe75e222eb43, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:57.288694 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:57.296178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1236/2450] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:36:57 2022] PUT /identity/v3/projects/ab13f7c8ebd44e639e4779c9647bd90d/users/3bf7d765887948e184efbe75e222eb43/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:37:02.772967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:02.773472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:02.773573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:02.807873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] MFA Rules not processed for user `3bf7d765887948e184efbe75e222eb43`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:37:02.809131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:02.850223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:02.880365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1215/2451] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:37:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:37:02.895039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:02.896880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:03.035418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf804d-e41f-436b-8cc7-11ac49fb8b50 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:03.041501 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:03.042266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:03.042615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:03.043079 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:03.118546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:03.120201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:03.130499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1237/2452] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:37:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:37:12.914293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=12 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:13.125617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:41.765198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:41.766618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:41.834163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c49a8-0b4c-4c59-a0c1-42e6b6a561ca None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:41.839172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:41.839786 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:41.840070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:41.840348 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:41.842106 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:41.853338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:41.854904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:41.865419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1216/2453] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:37:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:37:51.860495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=32 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:38:13.567952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:13.630052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514d289-e9a5-4006-a343-6c5d82ad851e service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:13.634576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:13.635210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:13.635491 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:13.635772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] PATH_INFO: `/v3/users/3864d76d4c3b478bad9e5c3c7024a062` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:13.636953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] RBAC: Authorizing `identity:delete_user(user_id=3864d76d4c3b478bad9e5c3c7024a062)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:13.638893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:13.845062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3864d76d4c3b478bad9e5c3c7024a062'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:13.845726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3864d76d4c3b478bad9e5c3c7024a062'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:13.907136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3864d76d4c3b478bad9e5c3c7024a062'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:13.921337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3864d76d4c3b478bad9e5c3c7024a062'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:13.922029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3864d76d4c3b478bad9e5c3c7024a062'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:13.935616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1238/2454] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:38:13 2022] DELETE /identity/v3/users/3864d76d4c3b478bad9e5c3c7024a062 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:38:14.159991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:14.170962 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8ca4d-b546-4b6b-b1fc-9abdbe4d7015 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:14.228745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:14.229368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:14.229644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:14.229917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] PATH_INFO: `/v3/projects/f9befc2c90574dae9c2f9d087b909062` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:14.231063 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9befc2c90574dae9c2f9d087b909062)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:14.232999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:14.365494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9befc2c90574dae9c2f9d087b909062', 'request_id': 'req-29113e93-7f61-496a-b872-74c24745cd27'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.372211 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9befc2c90574dae9c2f9d087b909062', 'request_id': 'req-29113e93-7f61-496a-b872-74c24745cd27'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.377818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9befc2c90574dae9c2f9d087b909062', 'request_id': 'req-29113e93-7f61-496a-b872-74c24745cd27'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.476551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] The token cache is being invalidate because project f9befc2c90574dae9c2f9d087b909062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:38:14.476889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.477646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.479230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1217/2455] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:38:14 2022] DELETE /identity/v3/projects/f9befc2c90574dae9c2f9d087b909062 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:38:14.483243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:14.484699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:14.500236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdfbd5-ec26-4eef-ba4d-e037220b25df admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:14.559632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:14.560272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:14.560584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:14.560858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] PATH_INFO: `/v3/users/3bf7d765887948e184efbe75e222eb43` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:14.562011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf7d765887948e184efbe75e222eb43)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:14.563926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:14.584802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:38:14.784489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf7d765887948e184efbe75e222eb43'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.785822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf7d765887948e184efbe75e222eb43'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.852759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf7d765887948e184efbe75e222eb43'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.864850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf7d765887948e184efbe75e222eb43'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.865483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf7d765887948e184efbe75e222eb43'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:14.877872 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1239/2456] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:38:14 2022] DELETE /identity/v3/users/3bf7d765887948e184efbe75e222eb43 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:38:15.152030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:15.154915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:15.167678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29113e93-7f61-496a-b872-74c24745cd27 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:15.226862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:15.227506 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:15.227790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:15.228066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] PATH_INFO: `/v3/projects/ab13f7c8ebd44e639e4779c9647bd90d` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:15.229281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab13f7c8ebd44e639e4779c9647bd90d)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:15.231241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:15.312462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:38:15.324907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab13f7c8ebd44e639e4779c9647bd90d', 'request_id': 'req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:15.331539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab13f7c8ebd44e639e4779c9647bd90d', 'request_id': 'req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:15.337874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab13f7c8ebd44e639e4779c9647bd90d', 'request_id': 'req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:15.401153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] The token cache is being invalidate because project ab13f7c8ebd44e639e4779c9647bd90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:38:15.401437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:15.402064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:38:15.403604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1218/2457] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:38:15 2022] DELETE /identity/v3/projects/ab13f7c8ebd44e639e4779c9647bd90d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:38:19.711526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:19.712980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:19.770764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ef9f1-9304-49c2-97f9-0427e2c3f084 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:19.817718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:19.818334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:19.818615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:19.818891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:19.820664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:19.891639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:38:19.893887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:19.895348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:19.906142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1240/2458] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:38:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:38:21.083678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:21.085155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:21.097842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0990f-b234-4520-a6d3-560a0b2e5a5e admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:21.101607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad8e38-55eb-4176-b634-b484908896c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:21.102296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8e38-55eb-4176-b634-b484908896c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:21.102990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8e38-55eb-4176-b634-b484908896c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:21.102990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8e38-55eb-4176-b634-b484908896c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:21.104669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-78ad8e38-55eb-4176-b634-b484908896c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:21.118879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad8e38-55eb-4176-b634-b484908896c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:21.120310 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad8e38-55eb-4176-b634-b484908896c9 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:21.131754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1219/2459] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:38:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:38:24.158915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:24.160931 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:24.178044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2012347-a959-42bb-9de6-76ead5eeb481 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:24.184156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:24.184926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:24.185211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:24.185483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:24.187311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:24.206187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:24.209561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:24.234863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1241/2460] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:38:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:38:31.126820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=18,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:38:34.216546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=8 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:38:40.920006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1d537-6152-48ae-b0e4-837da096813f None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:40.920476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1d537-6152-48ae-b0e4-837da096813f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:40.920584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1d537-6152-48ae-b0e4-837da096813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:40.980242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3ca1d537-6152-48ae-b0e4-837da096813f None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:38:40.981492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3ca1d537-6152-48ae-b0e4-837da096813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:41.044681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1220/2461] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 27 20:38:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:38:41.176187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:38:41.177886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:38:41.268123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d368c-a0b5-4a7e-ad65-1399915304c6 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:38:41.273953 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20766090-057d-4252-8858-5ced2890cfcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:38:41.274578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20766090-057d-4252-8858-5ced2890cfcf service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:38:41.274862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20766090-057d-4252-8858-5ced2890cfcf service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:38:41.275144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20766090-057d-4252-8858-5ced2890cfcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:38:41.283611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20766090-057d-4252-8858-5ced2890cfcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:38:41.285222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20766090-057d-4252-8858-5ced2890cfcf service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:38:41.293431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1242/2462] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:38:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:38:51.041740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=44 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:38:51.290067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=41 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:40:41.241484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073848e-5617-4778-8ccd-5351fb4d686e None None] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:41.242767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571ec55-fd1f-46f3-b40a-216b7b141c4a None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:41.243192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073848e-5617-4778-8ccd-5351fb4d686e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:41.243295 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571ec55-fd1f-46f3-b40a-216b7b141c4a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:41.243396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073848e-5617-4778-8ccd-5351fb4d686e None None] PATH_INFO: `/` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:41.243484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571ec55-fd1f-46f3-b40a-216b7b141c4a None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:41.243578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1243/2463] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 27 20:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:40:41.244303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1221/2464] 10.222.0.47 () {54 vars in 912 bytes} [Wed Jul 27 20:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:40:41.249558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:41.250147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:41.250714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:41.250845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:41.251103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:41.251388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:41.291627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:40:41.291885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:40:41.292925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:41.293124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:41.360926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1244/2465] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:40:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:40:41.369312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1222/2466] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:40:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:40:41.388769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:41.390133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:41.397661 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:41.399166 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:41.403970 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff1756-ace3-4260-85e6-fe5819c84607 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:41.409121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a3f58-f73a-4aeb-9c58-26c109033f31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:41.409738 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a3f58-f73a-4aeb-9c58-26c109033f31 service placement] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:41.410014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a3f58-f73a-4aeb-9c58-26c109033f31 service placement] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:41.410284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a3f58-f73a-4aeb-9c58-26c109033f31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:41.411917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa1086-7f93-45e1-81f7-4f73fc532a74 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:41.417121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:41.417735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:41.418017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:41.418290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:41.418858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a3f58-f73a-4aeb-9c58-26c109033f31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:41.420347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a3f58-f73a-4aeb-9c58-26c109033f31 service placement] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:41.427430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:41.429280 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:41.430178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1245/2467] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:41.439515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1223/2468] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:40:45.802372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5c2b0-bec6-472f-b263-16f343dc6380 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:45.802917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5c2b0-bec6-472f-b263-16f343dc6380 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:45.803026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5c2b0-bec6-472f-b263-16f343dc6380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:45.833587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-69b5c2b0-bec6-472f-b263-16f343dc6380 None None] MFA Rules not processed for user `caca4eade01642af9322aa72f7443726`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:40:45.835130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-69b5c2b0-bec6-472f-b263-16f343dc6380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:45.893725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1246/2469] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:40:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:40:46.196033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:46.205686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c29ddc-0815-4b7d-b0aa-687e8ea3a1fd service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:46.209326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:46.209954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:46.210236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:46.210509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:46.212326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:46.234276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:46.235699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:46.244185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1224/2470] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:46.632465 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5c2b0-bec6-472f-b263-16f343dc6380 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:46.644400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5c2b0-bec6-472f-b263-16f343dc6380 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:46.649359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:46.649979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:46.650263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:46.650549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:46.652290 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:46.665461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:46.667014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:46.678841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1247/2471] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:55.006531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:55.019805 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbe86a-413b-47d0-81c1-f014c5838310 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:55.029946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:55.030574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:55.030859 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:55.031135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:55.033189 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:55.046494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:55.049110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:55.059222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1225/2472] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:56.673402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=75 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:41:05.054579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=58 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:41:18.645342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:41:18.647287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:18.662183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4721f0-033e-4aee-9dff-bf9dab9be120 service placement] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:41:18.667517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:41:18.668143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:41:18.668450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:41:18.668724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:41:18.670378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:18.738460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:41:18.740055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:41:18.749243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1248/2473] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:41:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:41:18.809046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:41:18.820699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaceca-d9d9-48a1-886a-1bb7452a88fa service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:41:18.826316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:41:18.826933 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:41:18.827331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:41:18.827613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:41:18.829407 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:18.899567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:41:18.901123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:41:18.910282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1226/2474] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:41:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:41:28.745024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=33 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:41:28.905884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=33 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:41:35.762745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:41:35.774073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea26a2-0509-4f06-a725-72183f0f19fc service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:41:35.779569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:41:35.780168 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:41:35.780508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:41:35.780770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:41:35.782572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:35.806232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:41:35.807775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:41:35.816577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1249/2475] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:41:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:41:45.812672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=11 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:41:46.365163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:41:46.374939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a67de6-fa16-4e9d-9b03-9600cd8c4269 service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:41:46.378654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:41:46.379273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:41:46.379559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:41:46.379858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:41:46.381625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:46.391705 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:41:46.393160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:41:46.401802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1227/2476] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:41:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:41:56.397588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:42:41.234691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:42:41.236672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:42:41.351144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8f2ed-99d7-43eb-9d73-8d8d2980c456 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:42:41.363693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:42:41.364594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:42:41.365045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:42:41.365487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:42:41.371384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:42:41.512748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:42:41.514201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:42:41.523419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1250/2477] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:42:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:42:42.104937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:42:42.114209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86134b81-21a6-4cee-bb14-df671c549bf7 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:42:42.117992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:42:42.118601 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:42:42.119137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:42:42.119908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:42:42.121700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] Could not find project: 55c519d693b2424b9e0acc02143c6e6a.: keystone.exception.ProjectNotFound: Could not find project: 55c519d693b2424b9e0acc02143c6e6a. Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 27 20:42:42.134297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:42:42.136751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:42:42.138821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application with Lock( Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 20:42:42.140663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 20:42:42.142536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 55c519d693b2424b9e0acc02143c6e6a. Jul 27 20:42:42.142536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: ERROR keystone.server.flask.application Jul 27 20:42:42.142536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1228/2478] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:42:42 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 27 20:42:42.579203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:42:42.608811 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49042921-3e2a-4b14-be12-75b8c791c12e service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:42:42.614179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:42:42.615184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:42:42.615500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:42:42.616543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:42:42.618002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:42:42.662941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:42:42.664564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:42:42.675690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1251/2479] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:42:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:42:52.125323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:42:52.670881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=85 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:43:21.358022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:43:21.387897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafe347-9ab3-4123-9e5e-f4eb547b9e0d service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:43:21.391533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f506c-6752-4246-95ae-05b8f24d3c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:43:21.392139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f506c-6752-4246-95ae-05b8f24d3c79 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:43:21.392445 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f506c-6752-4246-95ae-05b8f24d3c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:43:21.392727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f506c-6752-4246-95ae-05b8f24d3c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:43:21.409442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f506c-6752-4246-95ae-05b8f24d3c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:43:21.410825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f506c-6752-4246-95ae-05b8f24d3c79 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:43:21.434795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1229/2480] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:43:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:43:22.542348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:43:22.562016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147ae3-22a7-4568-9346-caf5ec554f4e service placement] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:43:22.565766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:43:22.566396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:43:22.566672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:43:22.566943 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:43:22.580087 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:43:22.581529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:43:22.592601 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1252/2481] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:43:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:43:31.431492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=22 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:43:32.587439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=11 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:44:25.451889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a9095-39ba-4f7a-824e-5bfc85d0c844 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:25.452459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a9095-39ba-4f7a-824e-5bfc85d0c844 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:25.452459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a9095-39ba-4f7a-824e-5bfc85d0c844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:25.479926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c60a9095-39ba-4f7a-824e-5bfc85d0c844 None None] MFA Rules not processed for user `6cc262a3c3314be8a6ee620ebfa6fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:44:25.481236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c60a9095-39ba-4f7a-824e-5bfc85d0c844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:25.488187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1230/2482] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:44:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:44:25.786128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:25.795055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74c452-1e7f-4401-bab2-500f6b4d8221 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:25.798700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:25.799303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:25.799584 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:25.799860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:25.801755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:25.811799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:25.813273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:25.821847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1253/2483] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:44:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:44:29.905595 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a9095-39ba-4f7a-824e-5bfc85d0c844 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:29.920362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a9095-39ba-4f7a-824e-5bfc85d0c844 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:29.927408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bd611-f3be-4839-a5f7-7e692d6672aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:29.928138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bd611-f3be-4839-a5f7-7e692d6672aa service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:29.928667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bd611-f3be-4839-a5f7-7e692d6672aa service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:29.928947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bd611-f3be-4839-a5f7-7e692d6672aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:29.939468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7bd611-f3be-4839-a5f7-7e692d6672aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:29.941227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7bd611-f3be-4839-a5f7-7e692d6672aa service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:29.953686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1231/2484] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:44:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:44:35.817707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:44:39.948777 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=12 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:44:48.661626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:48.670786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e616662-c3a7-4f38-86cf-523caf5f26ba service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:48.674386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4002cd73-8d8e-4f94-b718-867f83a8d60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:48.674983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002cd73-8d8e-4f94-b718-867f83a8d60e service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:48.675260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002cd73-8d8e-4f94-b718-867f83a8d60e service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:48.675528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4002cd73-8d8e-4f94-b718-867f83a8d60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:48.677264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4002cd73-8d8e-4f94-b718-867f83a8d60e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:48.738412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002cd73-8d8e-4f94-b718-867f83a8d60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:48.739830 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4002cd73-8d8e-4f94-b718-867f83a8d60e service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:48.749748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1254/2485] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:44:52.633645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842687f6-d472-405c-b61a-11521cf78a75 None None] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:52.634146 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842687f6-d472-405c-b61a-11521cf78a75 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:52.634350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842687f6-d472-405c-b61a-11521cf78a75 None None] PATH_INFO: `/` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:52.635807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1232/2486] 10.222.0.47 () {54 vars in 921 bytes} [Wed Jul 27 20:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:44:52.666070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:52.666616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:52.666616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:52.688359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] MFA Rules not processed for user `328ec1aac43349b28a72d5df19fd2cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:44:52.689575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:52.699197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1255/2487] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Jul 27 20:44:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:44:52.710122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842687f6-d472-405c-b61a-11521cf78a75 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:52.723948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842687f6-d472-405c-b61a-11521cf78a75 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:52.730074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:52.732798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:52.732798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:52.732798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:52.739954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:52.741539 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:52.753039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1233/2488] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:44:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:44:52.767809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:52.768178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:52.793711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479582f5-1360-4cd3-9d0f-f569c08d2f36 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:52.799109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:52.799759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:52.800049 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:52.800324 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] PATH_INFO: `/v3/users/77b0df6c83a84dbcb0fa5802c93dc6ae` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:52.801672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b0df6c83a84dbcb0fa5802c93dc6ae)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:52.803587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:52.976226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b0df6c83a84dbcb0fa5802c93dc6ae'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:52.976899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b0df6c83a84dbcb0fa5802c93dc6ae'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.054840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b0df6c83a84dbcb0fa5802c93dc6ae'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.067700 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b0df6c83a84dbcb0fa5802c93dc6ae'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.068366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-707b5a10-7c31-4a83-8669-94ef27938862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b0df6c83a84dbcb0fa5802c93dc6ae'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.081870 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1256/2489] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:44:52 2022] DELETE /identity/v3/users/77b0df6c83a84dbcb0fa5802c93dc6ae => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:44:53.318709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:53.332291 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ceb92f-7f45-49a5-81b2-31e65b889493 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:53.391971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:53.392644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:53.392926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:53.393204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] PATH_INFO: `/v3/projects/2ff46475bc95496f93ad4ac4459d7cc5` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:53.394359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ff46475bc95496f93ad4ac4459d7cc5)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:53.396282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:53.516945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ff46475bc95496f93ad4ac4459d7cc5', 'request_id': 'req-bdfc88be-18b6-4ded-913a-277583bce9d6'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.523185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ff46475bc95496f93ad4ac4459d7cc5', 'request_id': 'req-bdfc88be-18b6-4ded-913a-277583bce9d6'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.528791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ff46475bc95496f93ad4ac4459d7cc5', 'request_id': 'req-bdfc88be-18b6-4ded-913a-277583bce9d6'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.589992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] The token cache is being invalidate because project 2ff46475bc95496f93ad4ac4459d7cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:44:53.590221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.590850 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bdfc88be-18b6-4ded-913a-277583bce9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:44:53.592439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1234/2490] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:44:53 2022] DELETE /identity/v3/projects/2ff46475bc95496f93ad4ac4459d7cc5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:44:53.599095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fdbba-6ccb-434f-b554-21397253e746 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:53.599440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fdbba-6ccb-434f-b554-21397253e746 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:53.599747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fdbba-6ccb-434f-b554-21397253e746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:53.619912 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-372fdbba-6ccb-434f-b554-21397253e746 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:44:53.621206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-372fdbba-6ccb-434f-b554-21397253e746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:53.674043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:44:53.682516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1257/2491] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:44:53.690403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c061391-c22f-4943-b7a4-2958e429880b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:53.690870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c061391-c22f-4943-b7a4-2958e429880b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:53.690979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c061391-c22f-4943-b7a4-2958e429880b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:53.711463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-5c061391-c22f-4943-b7a4-2958e429880b None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:44:53.712725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-5c061391-c22f-4943-b7a4-2958e429880b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:53.720733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1235/2492] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:44:53.724852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fdbba-6ccb-434f-b554-21397253e746 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:53.734765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fdbba-6ccb-434f-b554-21397253e746 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:53.739445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:53.740053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:53.740363 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:53.740667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:53.741362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:44:53.741668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:53.743913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:53.755991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1258/2493] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:44:53.760577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c061391-c22f-4943-b7a4-2958e429880b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:53.769825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c061391-c22f-4943-b7a4-2958e429880b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:53.775017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:53.775613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:53.775896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:53.776170 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:53.776880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:53.778787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:53.862297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1236/2494] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:44:53 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:44:53.866647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:53.877493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d39595-3835-4675-b29e-ee8472cf78c2 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:53.932560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:53.933161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:53.933440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:53.933711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:53.934360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:53.936235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:54.014639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1259/2495] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:44:53 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:44:54.018691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:54.029047 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0571b67-9083-417c-bdc7-1a2e1204527a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:54.033695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:54.034309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:54.034589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:54.034861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:54.035580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:54.037186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:54.047017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1237/2496] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:44:54.051141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:54.061945 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b369a-3476-4870-96ab-93254c3f6e8f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:54.066589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190516f6-2840-4625-9a30-cbe0417fa821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:54.067199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190516f6-2840-4625-9a30-cbe0417fa821 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:54.067482 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190516f6-2840-4625-9a30-cbe0417fa821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:54.067762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190516f6-2840-4625-9a30-cbe0417fa821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:54.068541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190516f6-2840-4625-9a30-cbe0417fa821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:54.071445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190516f6-2840-4625-9a30-cbe0417fa821 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:54.081726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1260/2497] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:44:54.086043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:54.096039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22685-762c-4b59-91d9-9b36d052bb8d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:54.101073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:54.101737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:54.102019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:54.102299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] PATH_INFO: `/v3/projects/345a49c97edc4fb7b1ce7126ce48b5b5/users/3b94ad3c2863476ea5067d09818f66ba/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:54.120614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345a49c97edc4fb7b1ce7126ce48b5b5, user_id=3b94ad3c2863476ea5067d09818f66ba, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:54.122264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:54.180616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1238/2498] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:44:54 2022] PUT /identity/v3/projects/345a49c97edc4fb7b1ce7126ce48b5b5/users/3b94ad3c2863476ea5067d09818f66ba/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:44:54.186747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2512d-c0fd-4855-bb7c-6e50cdfc324a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:54.187072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2512d-c0fd-4855-bb7c-6e50cdfc324a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:54.187347 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2512d-c0fd-4855-bb7c-6e50cdfc324a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:54.212621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-e2a2512d-c0fd-4855-bb7c-6e50cdfc324a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:44:54.213868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e2a2512d-c0fd-4855-bb7c-6e50cdfc324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:54.272170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1261/2499] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:44:54.281948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:54.283775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:54.298680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:44:54.467743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ee49d-8c0e-4cb7-b530-bb56f62fead4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:54.471592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:54.472199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:54.476963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:54.476963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:54.482368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:54.483822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:54.494176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1239/2500] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:44:59.634273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df638c-0d76-4c63-8cb5-2140f796d18b None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:59.634797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df638c-0d76-4c63-8cb5-2140f796d18b None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:59.634949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df638c-0d76-4c63-8cb5-2140f796d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:59.665070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d8df638c-0d76-4c63-8cb5-2140f796d18b None None] MFA Rules not processed for user `3b94ad3c2863476ea5067d09818f66ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:44:59.666321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d8df638c-0d76-4c63-8cb5-2140f796d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:59.724285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:44:59.733642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1262/2501] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:44:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:44:59.749968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:44:59.751905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:44:59.830485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a04e-33fa-4ec1-add4-529f167c9c84 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:44:59.836084 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:44:59.836807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:44:59.837088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:44:59.837363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:44:59.846738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:44:59.847748 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:44:59.857999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1240/2502] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:44:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:02.513485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:02.514015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:02.514128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:02.533349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] MFA Rules not processed for user `3b94ad3c2863476ea5067d09818f66ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:45:02.534580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:02.542120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1263/2503] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:45:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:45:09.853603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=54 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:12.527789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:19.095016 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:19.106809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0e937-f571-485d-9bb4-35a757a0ad5c service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:19.112546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:19.116306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:19.116306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:19.116306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:19.116306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:19.177322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:19.178954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:19.188252 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1241/2504] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:46:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:46:22.191997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:22.193419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:22.271990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63887cc-171d-48d2-bf50-930c6d13ab7a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:22.280697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:22.280697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:22.280697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:22.280697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:22.281120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:22.347927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:22.349414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:22.368152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1264/2505] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:46:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:46:29.183948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=28 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:32.365352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=65 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:43.578614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:43.580020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:43.591372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8dc9d-fe56-490d-89d5-4873f744eb0d service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:43.594825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:43.595416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:43.595691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:43.595958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:43.597646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:43.673212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:43.674636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:43.693143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1242/2506] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:46:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:46:50.375298 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:50.376894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:50.389169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4ee5e-0501-4e6e-b585-4b6d8eaa8383 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:50.392763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:50.393368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:50.393647 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:50.393920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:50.395614 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:50.405420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:50.406837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:50.415748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1265/2507] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:46:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:46:53.690453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=36 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:00.411357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:53.677385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.678785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:53.747822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e528f-9890-4f00-8236-04e2dd9b5d2f service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.752974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.753584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.754367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.754642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.756494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:53.767931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:53.770050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:53.780409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1243/2508] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:03.776175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=32 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:20.731167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:20.731167 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:20.811876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f7dc1-f10c-4c21-8083-d7b96c9da45a service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:20.815599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:20.816216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:20.816539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:20.816821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:20.818696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] Could not find project: f9befc2c90574dae9c2f9d087b909062.: keystone.exception.ProjectNotFound: Could not find project: f9befc2c90574dae9c2f9d087b909062. Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 27 20:48:20.832173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:48:20.835884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:48:20.840669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application with Lock( Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 20:48:20.842401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 20:48:20.843788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f9befc2c90574dae9c2f9d087b909062. Jul 27 20:48:20.843788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: ERROR keystone.server.flask.application Jul 27 20:48:20.843788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1266/2509] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:20 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 105 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 27 20:48:22.406372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:22.416290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166d4e-138d-447f-a28f-442631e85676 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:22.419883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:22.420517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:22.420800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:22.421075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:22.422825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:22.455866 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:22.457340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:22.476956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1244/2510] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:25.881677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:25.883208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:25.898440 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9997e5-12a7-4f93-b298-7f0b5ffc08b9 service glance] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:25.903020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:25.903669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:25.904073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:25.904486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] PATH_INFO: `/v3/users/caca4eade01642af9322aa72f7443726` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:25.905683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=caca4eade01642af9322aa72f7443726)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:25.907691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:26.146286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caca4eade01642af9322aa72f7443726'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.146846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caca4eade01642af9322aa72f7443726'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.218073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caca4eade01642af9322aa72f7443726'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.230914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caca4eade01642af9322aa72f7443726'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.231546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafcfdb7-f250-4fa3-a7df-2bd95c4bc07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caca4eade01642af9322aa72f7443726'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.243638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1267/2511] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:25 2022] DELETE /identity/v3/users/caca4eade01642af9322aa72f7443726 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:26.492136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:26.503578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bc25f-951a-46aa-849b-248b0b882607 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:26.587477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:26.588125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:26.588644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:26.589109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] PATH_INFO: `/v3/projects/a74675480e734145a1692641342c3e99` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:26.590294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74675480e734145a1692641342c3e99)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:26.592362 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:26.694214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74675480e734145a1692641342c3e99', 'request_id': 'req-79ee18ee-bcb3-4436-a760-f1085347c3c8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.700488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74675480e734145a1692641342c3e99', 'request_id': 'req-79ee18ee-bcb3-4436-a760-f1085347c3c8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.705971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74675480e734145a1692641342c3e99', 'request_id': 'req-79ee18ee-bcb3-4436-a760-f1085347c3c8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.755103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] The token cache is being invalidate because project a74675480e734145a1692641342c3e99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:26.755398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.756020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-79ee18ee-bcb3-4436-a760-f1085347c3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:26.757741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1245/2512] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:26 2022] DELETE /identity/v3/projects/a74675480e734145a1692641342c3e99 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:26.763450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71f4d-dc9f-4198-9377-1709af959485 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:26.763747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71f4d-dc9f-4198-9377-1709af959485 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:26.764029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71f4d-dc9f-4198-9377-1709af959485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:26.842004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f8c71f4d-dc9f-4198-9377-1709af959485 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:26.843213 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f8c71f4d-dc9f-4198-9377-1709af959485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:26.869899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:26.909960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1268/2513] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:26.974856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27f871-ff0d-428e-8c7c-8c536648444a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:26.975149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27f871-ff0d-428e-8c7c-8c536648444a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:26.975431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27f871-ff0d-428e-8c7c-8c536648444a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:26.996202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1d27f871-ff0d-428e-8c7c-8c536648444a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:26.997392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1d27f871-ff0d-428e-8c7c-8c536648444a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:27.005978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1246/2514] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:27.010339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c71f4d-dc9f-4198-9377-1709af959485 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.019869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c71f4d-dc9f-4198-9377-1709af959485 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.024360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.024997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.025274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.025545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.026258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:48:27.026557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.028143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.039409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1269/2515] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:48:27.043749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27f871-ff0d-428e-8c7c-8c536648444a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.054009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27f871-ff0d-428e-8c7c-8c536648444a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.058787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.059386 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.059665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.059935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.060625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.062502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.151217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1247/2516] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:27 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:27.155532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.166540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aff99-b8b9-4912-8af0-f7821655fca5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.221615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.222225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.222507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.222779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.223445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.225922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.292979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1270/2517] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:27 2022] POST /identity/v3/users => generated 282 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:27.297427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.308819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4461a1-c921-4958-8c09-ed7ea82c6b85 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.313518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.314120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.314422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.314695 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.315390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.317008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.322394 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:27.327354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1248/2518] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:27.331622 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.341919 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cf7cf-30b3-4f47-aa37-1bf9a8b38413 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.346881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ff3e2-178b-4d3f-9052-581d1a1bc27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.350410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ff3e2-178b-4d3f-9052-581d1a1bc27b admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.350410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ff3e2-178b-4d3f-9052-581d1a1bc27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.350410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ff3e2-178b-4d3f-9052-581d1a1bc27b admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.350410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ff3e2-178b-4d3f-9052-581d1a1bc27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.350995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ff3e2-178b-4d3f-9052-581d1a1bc27b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.362180 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1271/2519] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:27.367198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.377045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ab0b9-3a83-45b3-829c-2f9c96a7601f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.381845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.382535 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.382854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.383137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] PATH_INFO: `/v3/projects/8df4393a209f428bbd47cd279761c8a9/users/93ae55483b5045e78d24eb1277e8b737/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.397905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df4393a209f428bbd47cd279761c8a9, user_id=93ae55483b5045e78d24eb1277e8b737, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.402108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.469799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1249/2520] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:27 2022] PUT /identity/v3/projects/8df4393a209f428bbd47cd279761c8a9/users/93ae55483b5045e78d24eb1277e8b737/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:27.480632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1cf87-c6da-482f-92a3-9bc706c7b740 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.480915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1cf87-c6da-482f-92a3-9bc706c7b740 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.481192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1cf87-c6da-482f-92a3-9bc706c7b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.508575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-59a1cf87-c6da-482f-92a3-9bc706c7b740 None None] MFA Rules not processed for user `93ae55483b5045e78d24eb1277e8b737`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:27.509794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-59a1cf87-c6da-482f-92a3-9bc706c7b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:27.577330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1272/2521] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:27.577533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.650740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71458088-d47f-4804-9ad9-4ad87abdc1d9 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.650740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.650740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.650740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.651157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.651157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.651545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.739513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1250/2522] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:27 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:27.743753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1cf87-c6da-482f-92a3-9bc706c7b740 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.754548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1cf87-c6da-482f-92a3-9bc706c7b740 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.774866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:27.811579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.812206 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.812613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.812895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.813594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.815508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.921782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1273/2523] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:27 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:48:27.926035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.937130 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dccd8-6254-46f8-bc11-500044285df9 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.941903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.945021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.945529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.945810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.946526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.948158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:27.959815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1251/2524] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:27.964130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:27.974757 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50350-be60-46cd-9043-4ce2a726b883 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:27.981606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:27.981606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:27.981606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:27.981606 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] PATH_INFO: `/v3/projects/06bda4f1fb23428780958a51c71715d7/users/27d15177b8d84f1c9d2cc1088198019c/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:27.995551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bda4f1fb23428780958a51c71715d7, user_id=27d15177b8d84f1c9d2cc1088198019c, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:27.998210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:28.087432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1274/2525] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:27 2022] PUT /identity/v3/projects/06bda4f1fb23428780958a51c71715d7/users/27d15177b8d84f1c9d2cc1088198019c/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:28.091409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:28.101562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63502a-1dcf-4141-81a4-bf808c3815bc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:28.160335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6793d48-6067-45be-82c2-e4a405185ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:28.160991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793d48-6067-45be-82c2-e4a405185ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:28.161267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793d48-6067-45be-82c2-e4a405185ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:28.161544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793d48-6067-45be-82c2-e4a405185ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:28.162231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6793d48-6067-45be-82c2-e4a405185ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:28.163825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6793d48-6067-45be-82c2-e4a405185ad0 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:28.173602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1252/2526] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:28.178422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:28.189012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e7879-e7d8-46fd-abe3-9931fe74b96e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:28.194279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:28.194964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:28.198487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:28.198487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] PATH_INFO: `/v3/projects/06bda4f1fb23428780958a51c71715d7/users/27d15177b8d84f1c9d2cc1088198019c/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:28.198487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bda4f1fb23428780958a51c71715d7, user_id=27d15177b8d84f1c9d2cc1088198019c, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:28.200212 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:28.210695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1275/2527] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:28 2022] PUT /identity/v3/projects/06bda4f1fb23428780958a51c71715d7/users/27d15177b8d84f1c9d2cc1088198019c/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:28.221683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:28.221972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:28.222248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:28.250121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] MFA Rules not processed for user `27d15177b8d84f1c9d2cc1088198019c`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:28.251403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:28.286128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:28.328395 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1253/2528] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:28.347464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:28.347464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:28.405252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eda75-3ed1-4ede-a0b9-0a097966fad4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:28.409062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:28.409696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:28.409983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:28.410257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:28.421568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:28.423089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:28.432873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1276/2529] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:28.661114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:28.662749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:28.675568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f369b10-ede8-41c0-a736-d3b2bbcbbe47 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:28.679424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84d6ca-808c-49fa-9767-94382340d52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:28.680041 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84d6ca-808c-49fa-9767-94382340d52f service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:28.680322 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84d6ca-808c-49fa-9767-94382340d52f service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:28.680624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84d6ca-808c-49fa-9767-94382340d52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:28.737096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84d6ca-808c-49fa-9767-94382340d52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:28.738528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84d6ca-808c-49fa-9767-94382340d52f service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:28.747616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1254/2530] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:33.289005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:33.352839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672ac48-ec29-485a-bbcb-02ce64becd14 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:33.357566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:33.358253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:33.358532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:33.358806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] PATH_INFO: `/v3/users/93ae55483b5045e78d24eb1277e8b737` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:33.359988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ae55483b5045e78d24eb1277e8b737)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:33.361929 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:33.384874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:33.499797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ae55483b5045e78d24eb1277e8b737'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:33.500473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ae55483b5045e78d24eb1277e8b737'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:33.572245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ae55483b5045e78d24eb1277e8b737'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:33.584252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ae55483b5045e78d24eb1277e8b737'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:33.584923 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ae55483b5045e78d24eb1277e8b737'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:33.597092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1277/2531] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:33 2022] DELETE /identity/v3/users/93ae55483b5045e78d24eb1277e8b737 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:33.601666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:33.602144 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:33.602247 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:33.621366 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:33.622591 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:33.678979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1255/2532] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:48:33.690560 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:33.691961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:33.747586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdf6ec-af25-4f92-a21a-1ba12dcee99a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:33.751220 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:33.751839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:33.752116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:33.752400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:33.762631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:33.764236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:33.775169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1278/2533] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:34.448798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:34.450212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:34.462885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2737cbb-fce9-466c-aeef-b4ca50906df1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:34.468414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:34.469035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:34.469373 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:34.469644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] PATH_INFO: `/v3/projects/8df4393a209f428bbd47cd279761c8a9` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:34.477489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df4393a209f428bbd47cd279761c8a9)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:34.477489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:34.573907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df4393a209f428bbd47cd279761c8a9', 'request_id': 'req-daf9c4e9-4851-4fbe-8960-38ec2d858d82'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.581284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df4393a209f428bbd47cd279761c8a9', 'request_id': 'req-daf9c4e9-4851-4fbe-8960-38ec2d858d82'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.588056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df4393a209f428bbd47cd279761c8a9', 'request_id': 'req-daf9c4e9-4851-4fbe-8960-38ec2d858d82'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.590547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:34.664551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] The token cache is being invalidate because project 8df4393a209f428bbd47cd279761c8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:34.665180 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.665476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.667114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1256/2534] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:34 2022] DELETE /identity/v3/projects/8df4393a209f428bbd47cd279761c8a9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:34.670987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:34.682393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d7cfc-9784-43ce-b08e-7264015e9c44 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:34.743240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:34.743876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:34.744157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:34.744467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] PATH_INFO: `/v3/users/27d15177b8d84f1c9d2cc1088198019c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:34.745632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d15177b8d84f1c9d2cc1088198019c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:34.747552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:34.932983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:34.935265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d15177b8d84f1c9d2cc1088198019c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.935908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d15177b8d84f1c9d2cc1088198019c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.977704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d15177b8d84f1c9d2cc1088198019c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.990824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d15177b8d84f1c9d2cc1088198019c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:34.991571 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-95af0783-50c5-46c2-a855-6027f44bdb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d15177b8d84f1c9d2cc1088198019c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:35.006289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1279/2535] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:34 2022] DELETE /identity/v3/users/27d15177b8d84f1c9d2cc1088198019c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:35.374580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:35.376036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:35.397251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9c4e9-4851-4fbe-8960-38ec2d858d82 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:35.458741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:35.459393 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:35.459676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:35.459949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] PATH_INFO: `/v3/projects/06bda4f1fb23428780958a51c71715d7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:35.461183 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06bda4f1fb23428780958a51c71715d7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:35.463220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:35.568345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06bda4f1fb23428780958a51c71715d7', 'request_id': 'req-1744bb64-3ff7-47c1-96c4-320e89fe694f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:35.575085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06bda4f1fb23428780958a51c71715d7', 'request_id': 'req-1744bb64-3ff7-47c1-96c4-320e89fe694f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:35.581316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06bda4f1fb23428780958a51c71715d7', 'request_id': 'req-1744bb64-3ff7-47c1-96c4-320e89fe694f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:35.640350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] The token cache is being invalidate because project 06bda4f1fb23428780958a51c71715d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:35.640642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:35.641265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1744bb64-3ff7-47c1-96c4-320e89fe694f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:35.642833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1257/2536] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:35 2022] DELETE /identity/v3/projects/06bda4f1fb23428780958a51c71715d7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:35.647879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892e24e-3939-43eb-b13c-503ba257d546 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:35.648169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892e24e-3939-43eb-b13c-503ba257d546 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:35.648603 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892e24e-3939-43eb-b13c-503ba257d546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:35.669207 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-1892e24e-3939-43eb-b13c-503ba257d546 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:35.670434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1892e24e-3939-43eb-b13c-503ba257d546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:35.733851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1280/2537] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:48:35.742027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f56b-b239-4a98-86d8-e10cc20f2a6d None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:35.742511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f56b-b239-4a98-86d8-e10cc20f2a6d None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:35.742988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f56b-b239-4a98-86d8-e10cc20f2a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:35.765195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-dc44f56b-b239-4a98-86d8-e10cc20f2a6d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:35.766575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dc44f56b-b239-4a98-86d8-e10cc20f2a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:35.779176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1258/2538] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:35.780526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892e24e-3939-43eb-b13c-503ba257d546 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:35.794669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892e24e-3939-43eb-b13c-503ba257d546 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:35.800159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:35.800832 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:35.812138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:35.812138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:35.812138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:48:35.812138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:35.812138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:35.821341 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1281/2539] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:48:35.826038 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44f56b-b239-4a98-86d8-e10cc20f2a6d None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:35.836414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44f56b-b239-4a98-86d8-e10cc20f2a6d None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:35.841442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:35.842049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:35.842326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:35.842599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:35.843324 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:35.845253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:35.934685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1259/2540] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:35 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:35.938604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:35.949651 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341841d9-9d2b-4f39-9458-7f9da82eb754 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:36.009903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:36.010526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:36.010808 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:36.011082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:36.011745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:36.013701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:36.413037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1282/2541] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:35 2022] POST /identity/v3/users => generated 291 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:36.417076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:36.427682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079b131-8aac-450c-bb4b-01ed15853ba1 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:36.432236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:36.432871 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:36.433152 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:36.433425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:36.434119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:36.435700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:36.446194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1260/2542] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:36.450089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:36.460126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a00bd-9649-4bb0-9268-6997570ac618 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:36.464944 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336aba7e-6313-4ae9-a07d-8486f0ec379a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:36.465550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aba7e-6313-4ae9-a07d-8486f0ec379a admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:36.465831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aba7e-6313-4ae9-a07d-8486f0ec379a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:36.466109 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336aba7e-6313-4ae9-a07d-8486f0ec379a admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:36.466797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336aba7e-6313-4ae9-a07d-8486f0ec379a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:36.468486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336aba7e-6313-4ae9-a07d-8486f0ec379a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:36.481266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1283/2543] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:36.485686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:36.504485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d136b-e310-4cae-8bc5-f6ac1f00e7a2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:36.512691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:36.513542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:36.513904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:36.514220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] PATH_INFO: `/v3/projects/0c80923c587d4a8194e4e3796dbd30ef/users/f180ca81c3d94f4d9940e58f67b22ecf/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:36.529031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c80923c587d4a8194e4e3796dbd30ef, user_id=f180ca81c3d94f4d9940e58f67b22ecf, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:36.531696 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:36.593263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1261/2544] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:36 2022] PUT /identity/v3/projects/0c80923c587d4a8194e4e3796dbd30ef/users/f180ca81c3d94f4d9940e58f67b22ecf/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:48:36.600510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f688d8-152a-47c3-970d-d8c71eb94830 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:36.600799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f688d8-152a-47c3-970d-d8c71eb94830 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:36.601082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f688d8-152a-47c3-970d-d8c71eb94830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:36.630372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-89f688d8-152a-47c3-970d-d8c71eb94830 None None] MFA Rules not processed for user `f180ca81c3d94f4d9940e58f67b22ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:36.631626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-89f688d8-152a-47c3-970d-d8c71eb94830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:36.644349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:36.697439 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1284/2545] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:36.701911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:36.724489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:36.766131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4532d8-d8d7-4e5e-abc8-ebeb9aab4379 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:36.770981 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:36.771599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:36.771885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:36.772161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:36.772890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:36.774865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:36.855000 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1262/2546] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:36 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:36.857071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f688d8-152a-47c3-970d-d8c71eb94830 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:36.869323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f688d8-152a-47c3-970d-d8c71eb94830 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:36.930576 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:36.931195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:36.931477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:36.931752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:36.932471 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:36.934391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:37.025617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1285/2547] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:36 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:37.029224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:37.039914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d572db2-51c1-41cd-a875-2a561d30845b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:37.045698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:37.046326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.046606 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.046879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.047574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:37.049187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:37.060353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1263/2548] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:37.064730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:37.074955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce1dba-1850-4ada-bfc0-b2eaaa07eed8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:37.082253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:37.082253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.082253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.082253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] PATH_INFO: `/v3/projects/052b2842fc544b77a9c4c6c7b393c9e5/users/b0bd6748ad354e92ac332728c8eae512/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.099906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=052b2842fc544b77a9c4c6c7b393c9e5, user_id=b0bd6748ad354e92ac332728c8eae512, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:37.104735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:37.190903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1286/2549] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:37 2022] PUT /identity/v3/projects/052b2842fc544b77a9c4c6c7b393c9e5/users/b0bd6748ad354e92ac332728c8eae512/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:48:37.194970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:37.205160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678882b4-21ef-4a86-aec0-af2cabcb7a33 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:37.262294 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbbd1f-4702-43b4-8170-026f4b0f45bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:37.262907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbd1f-4702-43b4-8170-026f4b0f45bb admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.263185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbd1f-4702-43b4-8170-026f4b0f45bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.263461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbd1f-4702-43b4-8170-026f4b0f45bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.264156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbbd1f-4702-43b4-8170-026f4b0f45bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:37.265882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbbd1f-4702-43b4-8170-026f4b0f45bb admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:37.279792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1264/2550] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:37.284279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:37.296003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d664a3-5942-4f8c-8195-1037acb5fcf2 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:37.300915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:37.301893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.302416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.302894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] PATH_INFO: `/v3/projects/052b2842fc544b77a9c4c6c7b393c9e5/users/b0bd6748ad354e92ac332728c8eae512/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.306330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=052b2842fc544b77a9c4c6c7b393c9e5, user_id=b0bd6748ad354e92ac332728c8eae512, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:37.310570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:37.326333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1287/2551] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:37 2022] PUT /identity/v3/projects/052b2842fc544b77a9c4c6c7b393c9e5/users/b0bd6748ad354e92ac332728c8eae512/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:37.342442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bb36e-b8ee-4f45-885a-777e834c5f93 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.342442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bb36e-b8ee-4f45-885a-777e834c5f93 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.342442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bb36e-b8ee-4f45-885a-777e834c5f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.373545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-786bb36e-b8ee-4f45-885a-777e834c5f93 None None] MFA Rules not processed for user `b0bd6748ad354e92ac332728c8eae512`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:37.374677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-786bb36e-b8ee-4f45-885a-777e834c5f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:37.449481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1265/2552] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:37.460780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:37.462521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:37.521030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef51aa-8b09-4626-86c2-d27376a8d9c5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:37.524906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1420055-abd5-4db5-b1c7-ddcd4b7c2303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:37.525529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420055-abd5-4db5-b1c7-ddcd4b7c2303 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.525813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420055-abd5-4db5-b1c7-ddcd4b7c2303 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.526092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420055-abd5-4db5-b1c7-ddcd4b7c2303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.535629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1420055-abd5-4db5-b1c7-ddcd4b7c2303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:37.537636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1420055-abd5-4db5-b1c7-ddcd4b7c2303 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:37.547384 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1288/2553] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:37.652091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bb36e-b8ee-4f45-885a-777e834c5f93 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:37.661503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bb36e-b8ee-4f45-885a-777e834c5f93 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:37.665670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:37.666305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.666583 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.667218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.679586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:37.724869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:37.726314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:37.735196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1266/2554] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:37.893710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:37.894205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:37.894329 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:37.914961 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] MFA Rules not processed for user `3b94ad3c2863476ea5067d09818f66ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:37.916188 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:37.921419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:37.972802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1289/2555] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:38.012908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:38.014437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:38.074630 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d97b0-cb36-4b33-b4fa-587adb9aaa6f service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:38.078367 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:38.079067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:38.079344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:38.079612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:38.081468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:38.166374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:38.167773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:38.180021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1267/2556] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:38.861086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:38.865064 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:38.877212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:38.879128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:38.925783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:38.935391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c706ca4-de29-4cd6-9c7e-d3b6846aa1e8 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:38.939702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd5240-303f-4c57-8694-f7169a5085be service glance] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:38.940769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:38.940946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01cebb-a538-4e98-8287-237ccaaf7bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:38.941411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:38.941571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01cebb-a538-4e98-8287-237ccaaf7bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:38.941695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:38.942433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01cebb-a538-4e98-8287-237ccaaf7bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:38.942607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:38.942700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01cebb-a538-4e98-8287-237ccaaf7bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:38.944304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:38.944479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-de01cebb-a538-4e98-8287-237ccaaf7bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:38.955854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:38.957463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:38.958480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01cebb-a538-4e98-8287-237ccaaf7bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:38.962491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01cebb-a538-4e98-8287-237ccaaf7bf2 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:38.967643 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1290/2557] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:38.971732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1268/2558] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:48:40.793608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:40.856065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659f290-35c2-4a3a-b7d6-963a326dfad5 service placement] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:40.861016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:40.861648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:40.861927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:40.862261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] PATH_INFO: `/v3/users/f180ca81c3d94f4d9940e58f67b22ecf` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:40.863415 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f180ca81c3d94f4d9940e58f67b22ecf)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:40.865627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:41.066328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f180ca81c3d94f4d9940e58f67b22ecf'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:41.066999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f180ca81c3d94f4d9940e58f67b22ecf'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:41.069373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:41.161936 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f180ca81c3d94f4d9940e58f67b22ecf'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:41.182675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f180ca81c3d94f4d9940e58f67b22ecf'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:41.183656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f180ca81c3d94f4d9940e58f67b22ecf'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:41.199938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1291/2559] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:40 2022] DELETE /identity/v3/users/f180ca81c3d94f4d9940e58f67b22ecf => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:41.204353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e183-7088-48ca-a972-7730d5760812 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:41.204896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e183-7088-48ca-a972-7730d5760812 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:41.205014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573e183-7088-48ca-a972-7730d5760812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:41.223864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-b573e183-7088-48ca-a972-7730d5760812 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:41.228524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b573e183-7088-48ca-a972-7730d5760812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:41.301131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1269/2560] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:41.312139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:41.313640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:41.373918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37b9da-f0ad-47bd-a451-9ca3fb3f7415 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:41.384884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:41.384884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:41.384884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:41.384884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:41.390395 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:41.391895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:41.404057 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1292/2561] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:41.802629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e183-7088-48ca-a972-7730d5760812 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:41.804303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b573e183-7088-48ca-a972-7730d5760812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:41.828737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573e183-7088-48ca-a972-7730d5760812 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:41.835212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:41.835969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:41.836350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:41.836877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] PATH_INFO: `/v3/projects/0c80923c587d4a8194e4e3796dbd30ef` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:41.838319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c80923c587d4a8194e4e3796dbd30ef)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:41.840698 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:41.978485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c80923c587d4a8194e4e3796dbd30ef', 'request_id': 'req-2c6b51f7-63ad-457e-85c6-5082390c50ff'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:41.985559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c80923c587d4a8194e4e3796dbd30ef', 'request_id': 'req-2c6b51f7-63ad-457e-85c6-5082390c50ff'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:41.992873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c80923c587d4a8194e4e3796dbd30ef', 'request_id': 'req-2c6b51f7-63ad-457e-85c6-5082390c50ff'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.075055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] The token cache is being invalidate because project 0c80923c587d4a8194e4e3796dbd30ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:42.081666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.081666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.089860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1270/2562] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:41 2022] DELETE /identity/v3/projects/0c80923c587d4a8194e4e3796dbd30ef => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:42.093762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:42.103921 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d43055-eae7-4b77-a8d2-4ab2e7bf363c service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:42.171374 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:42.172016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:42.172291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:42.172595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] PATH_INFO: `/v3/users/b0bd6748ad354e92ac332728c8eae512` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:42.173792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0bd6748ad354e92ac332728c8eae512)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:42.175766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:42.374973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0bd6748ad354e92ac332728c8eae512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.375680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0bd6748ad354e92ac332728c8eae512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.440949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0bd6748ad354e92ac332728c8eae512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.457038 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:42.458307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0bd6748ad354e92ac332728c8eae512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.459028 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-50c67fc3-9ad3-46fc-8de2-fc57480bacb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0bd6748ad354e92ac332728c8eae512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:42.487040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1293/2563] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:42 2022] DELETE /identity/v3/users/b0bd6748ad354e92ac332728c8eae512 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:42.979472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:42.981533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:43.003408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b51f7-63ad-457e-85c6-5082390c50ff admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:43.076693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:43.077359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.077644 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.078050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] PATH_INFO: `/v3/projects/052b2842fc544b77a9c4c6c7b393c9e5` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.081956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=052b2842fc544b77a9c4c6c7b393c9e5)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:43.088477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:43.097260 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:43.166272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052b2842fc544b77a9c4c6c7b393c9e5', 'request_id': 'req-f5356c85-41a8-4b3b-8a52-74993d69e0ff'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:43.172555 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052b2842fc544b77a9c4c6c7b393c9e5', 'request_id': 'req-f5356c85-41a8-4b3b-8a52-74993d69e0ff'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:43.179336 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052b2842fc544b77a9c4c6c7b393c9e5', 'request_id': 'req-f5356c85-41a8-4b3b-8a52-74993d69e0ff'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:43.247565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] The token cache is being invalidate because project 052b2842fc544b77a9c4c6c7b393c9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:43.247882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:43.248549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f5356c85-41a8-4b3b-8a52-74993d69e0ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:43.250055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1271/2564] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:42 2022] DELETE /identity/v3/projects/052b2842fc544b77a9c4c6c7b393c9e5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:43.254950 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c771f-26b6-4ac8-95da-4ddc18c1e180 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.255456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c771f-26b6-4ac8-95da-4ddc18c1e180 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.256143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c771f-26b6-4ac8-95da-4ddc18c1e180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.275516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3b9c771f-26b6-4ac8-95da-4ddc18c1e180 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:43.276756 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3b9c771f-26b6-4ac8-95da-4ddc18c1e180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:43.341909 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1294/2565] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:48:43.350026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efff33-edec-4511-a4d8-b833796d3be1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.350343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efff33-edec-4511-a4d8-b833796d3be1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.350623 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efff33-edec-4511-a4d8-b833796d3be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.372669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-08efff33-edec-4511-a4d8-b833796d3be1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:43.373880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-08efff33-edec-4511-a4d8-b833796d3be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:43.382547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1272/2566] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:48:43.386911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c771f-26b6-4ac8-95da-4ddc18c1e180 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:43.397133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c771f-26b6-4ac8-95da-4ddc18c1e180 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:43.401980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:43.402602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.402887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.403164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.403902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:48:43.404821 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:43.406044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:43.418309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1295/2567] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:48:43.423137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efff33-edec-4511-a4d8-b833796d3be1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:43.433674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efff33-edec-4511-a4d8-b833796d3be1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:43.438401 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:43.439045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.439396 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.439671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.440350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:43.442273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:43.524026 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1273/2568] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:43 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:43.525688 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:43.544137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216be192-c25c-4356-a434-5300e869a44c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:43.603980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:43.604558 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.604848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.605124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.605799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:43.607740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:43.681720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1296/2569] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:43 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:43.686027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:43.697140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0567f-a4d5-4acd-8663-960e17bf0398 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:43.702668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:43.703289 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.703570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.703843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.704579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:43.706173 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:43.716448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1274/2570] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:43.720522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:43.731177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa185-295b-4409-865b-eebcbc809b75 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:43.735911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78db40-6eaa-43c5-9acc-ec9216874eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:43.736556 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78db40-6eaa-43c5-9acc-ec9216874eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.736840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78db40-6eaa-43c5-9acc-ec9216874eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.737127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78db40-6eaa-43c5-9acc-ec9216874eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.737813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78db40-6eaa-43c5-9acc-ec9216874eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:43.739704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78db40-6eaa-43c5-9acc-ec9216874eb4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:43.751460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1297/2571] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:43.755618 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:43.773939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e1771-af8d-453f-9e59-1c3f8e3e021e admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:43.778684 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:43.782988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.782988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.782988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] PATH_INFO: `/v3/projects/aaf72b78e06a436c86be7be4df23939d/users/4cc8a98e67ac46b98135d392d06a6100/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.795278 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf72b78e06a436c86be7be4df23939d, user_id=4cc8a98e67ac46b98135d392d06a6100, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:43.797927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:43.927202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1275/2572] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:43 2022] PUT /identity/v3/projects/aaf72b78e06a436c86be7be4df23939d/users/4cc8a98e67ac46b98135d392d06a6100/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:43.938198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116faa5e-ceaf-4fe2-9005-4c377f9b5301 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:43.938489 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116faa5e-ceaf-4fe2-9005-4c377f9b5301 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:43.938764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116faa5e-ceaf-4fe2-9005-4c377f9b5301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:43.967635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-116faa5e-ceaf-4fe2-9005-4c377f9b5301 None None] MFA Rules not processed for user `4cc8a98e67ac46b98135d392d06a6100`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:43.968904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-116faa5e-ceaf-4fe2-9005-4c377f9b5301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:44.010549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:44.032247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1298/2573] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:48:44.036946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:44.100720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdc1b1-8b42-4a6f-a8d2-edc303d9cce4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:44.105790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:44.106602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.106892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.107162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.109080 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:44.109997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:44.238214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1276/2574] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:44 2022] POST /identity/v3/projects => generated 389 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:44.242773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116faa5e-ceaf-4fe2-9005-4c377f9b5301 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:44.253920 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116faa5e-ceaf-4fe2-9005-4c377f9b5301 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:44.310514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:44.311172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.311396 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.311672 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.312327 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:44.314271 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:44.394716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1299/2575] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:44 2022] POST /identity/v3/users => generated 301 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:44.398857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:44.409187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c575-4f11-42f0-90d5-c8bd3e79acdd admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:44.414508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:44.415116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.415398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.415671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.416485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:44.418013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:44.429305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1277/2576] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:44.433493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:44.443308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b68894-0d17-4840-93fa-263e2130ad9a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:44.447841 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:44.448526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.448806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.449078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] PATH_INFO: `/v3/projects/57084a6043d94cc2b33c5a67219f6e49/users/fbc7c6ce8d94412199cab19d6ad646af/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.463702 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57084a6043d94cc2b33c5a67219f6e49, user_id=fbc7c6ce8d94412199cab19d6ad646af, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:44.466615 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:44.520059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1300/2577] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:44 2022] PUT /identity/v3/projects/57084a6043d94cc2b33c5a67219f6e49/users/fbc7c6ce8d94412199cab19d6ad646af/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:48:44.524066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:44.534300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fdb69-df29-4437-9442-3ef365bc5d57 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:44.543129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:44.597197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc8610-f46d-46ba-92dd-f81dc5df7458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:44.597813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc8610-f46d-46ba-92dd-f81dc5df7458 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.598092 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc8610-f46d-46ba-92dd-f81dc5df7458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.598360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc8610-f46d-46ba-92dd-f81dc5df7458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.599048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc8610-f46d-46ba-92dd-f81dc5df7458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:44.601231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc8610-f46d-46ba-92dd-f81dc5df7458 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:44.610891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1278/2578] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:44.615871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:44.626325 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0a351-8b14-4873-83c7-4ad5fa0de146 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:44.631108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:44.631769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.632069 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.632330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] PATH_INFO: `/v3/projects/57084a6043d94cc2b33c5a67219f6e49/users/fbc7c6ce8d94412199cab19d6ad646af/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.637744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57084a6043d94cc2b33c5a67219f6e49, user_id=fbc7c6ce8d94412199cab19d6ad646af, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:44.637744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:44.647709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1301/2579] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:44 2022] PUT /identity/v3/projects/57084a6043d94cc2b33c5a67219f6e49/users/fbc7c6ce8d94412199cab19d6ad646af/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:44.659960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69103533-0ec3-4ad8-8e83-e11603a5ea12 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.660237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69103533-0ec3-4ad8-8e83-e11603a5ea12 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.660589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69103533-0ec3-4ad8-8e83-e11603a5ea12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.689214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-69103533-0ec3-4ad8-8e83-e11603a5ea12 None None] MFA Rules not processed for user `fbc7c6ce8d94412199cab19d6ad646af`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:44.690426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-69103533-0ec3-4ad8-8e83-e11603a5ea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:44.757857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1279/2580] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:44.769025 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:44.770795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:44.834850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20488c70-1c33-4231-8b01-cd2db10aad86 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:44.838635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372184e-d075-4287-84ba-a5ddf7d8cabc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:44.839342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372184e-d075-4287-84ba-a5ddf7d8cabc service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:44.839661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372184e-d075-4287-84ba-a5ddf7d8cabc service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:44.839942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372184e-d075-4287-84ba-a5ddf7d8cabc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:44.850314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372184e-d075-4287-84ba-a5ddf7d8cabc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:44.851824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372184e-d075-4287-84ba-a5ddf7d8cabc service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:44.861809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1302/2581] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:44.998314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69103533-0ec3-4ad8-8e83-e11603a5ea12 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:45.063274 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69103533-0ec3-4ad8-8e83-e11603a5ea12 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:45.067720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:45.068926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:45.069201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:45.069465 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] PATH_INFO: `/v3/users/4cc8a98e67ac46b98135d392d06a6100` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:45.070628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc8a98e67ac46b98135d392d06a6100)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:45.072559 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:45.222726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:45.232040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc8a98e67ac46b98135d392d06a6100'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:45.237847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc8a98e67ac46b98135d392d06a6100'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:45.320306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc8a98e67ac46b98135d392d06a6100'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:45.340621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc8a98e67ac46b98135d392d06a6100'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:45.351719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc8a98e67ac46b98135d392d06a6100'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:45.358117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1280/2582] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:44 2022] DELETE /identity/v3/users/4cc8a98e67ac46b98135d392d06a6100 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:45.362586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:45.363068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:45.367059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:45.390156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:45.391366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:45.426884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:45.454453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1303/2583] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:45.467640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:45.469068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:45.527409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319ab77-860d-43c4-9bed-b74688927763 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:45.531122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:45.531987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:45.532545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:45.532825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:45.542213 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:45.543920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:45.554551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1281/2584] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:45.969031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:45.970320 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:45.998906 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3edd5a-14db-4fcf-a7af-e1fb1dd20647 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:46.003707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:46.004782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:46.005067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:46.005346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] PATH_INFO: `/v3/projects/aaf72b78e06a436c86be7be4df23939d` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:46.006542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf72b78e06a436c86be7be4df23939d)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:46.009042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:46.135162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf72b78e06a436c86be7be4df23939d', 'request_id': 'req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.141487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf72b78e06a436c86be7be4df23939d', 'request_id': 'req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.147416 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf72b78e06a436c86be7be4df23939d', 'request_id': 'req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.229378 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] The token cache is being invalidate because project aaf72b78e06a436c86be7be4df23939d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:46.229674 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.230338 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.231871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1304/2585] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:45 2022] DELETE /identity/v3/projects/aaf72b78e06a436c86be7be4df23939d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:46.236083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:46.246416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b804650-5ee5-49d9-beb2-5afb5bab64ad service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:46.303680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:46.304323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:46.304636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:46.304912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] PATH_INFO: `/v3/users/fbc7c6ce8d94412199cab19d6ad646af` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:46.306131 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc7c6ce8d94412199cab19d6ad646af)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:46.308070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:46.460462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc7c6ce8d94412199cab19d6ad646af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.461085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc7c6ce8d94412199cab19d6ad646af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.554285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc7c6ce8d94412199cab19d6ad646af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.556443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:46.567167 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc7c6ce8d94412199cab19d6ad646af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.574561 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-7784efad-b088-48c4-afc2-79ab2b2cab30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc7c6ce8d94412199cab19d6ad646af'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:46.580388 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1282/2586] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:46 2022] DELETE /identity/v3/users/fbc7c6ce8d94412199cab19d6ad646af => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:46.985630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:46.987741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:47.002917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0acda-9be1-4dbc-8ea6-b0f512c15950 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:47.063517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:47.064276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.064542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.064823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] PATH_INFO: `/v3/projects/57084a6043d94cc2b33c5a67219f6e49` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.065983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57084a6043d94cc2b33c5a67219f6e49)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:47.068594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:47.187291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57084a6043d94cc2b33c5a67219f6e49', 'request_id': 'req-99ec59e9-79ba-4ddf-920b-881993b92291'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:47.194464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57084a6043d94cc2b33c5a67219f6e49', 'request_id': 'req-99ec59e9-79ba-4ddf-920b-881993b92291'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:47.200567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57084a6043d94cc2b33c5a67219f6e49', 'request_id': 'req-99ec59e9-79ba-4ddf-920b-881993b92291'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:47.202579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:47.286163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] The token cache is being invalidate because project 57084a6043d94cc2b33c5a67219f6e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:47.286969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:47.287667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99ec59e9-79ba-4ddf-920b-881993b92291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:47.289328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1305/2587] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:46 2022] DELETE /identity/v3/projects/57084a6043d94cc2b33c5a67219f6e49 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:47.294546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69615ee6-26a0-4f84-8f3f-8c647c372e24 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.295046 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69615ee6-26a0-4f84-8f3f-8c647c372e24 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.295410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69615ee6-26a0-4f84-8f3f-8c647c372e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.316532 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-69615ee6-26a0-4f84-8f3f-8c647c372e24 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:47.317792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-69615ee6-26a0-4f84-8f3f-8c647c372e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:47.380538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1283/2588] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:47.389337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0d0c-ed52-4277-88a6-46f506d7544c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.389629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0d0c-ed52-4277-88a6-46f506d7544c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.389907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0d0c-ed52-4277-88a6-46f506d7544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.411009 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-514b0d0c-ed52-4277-88a6-46f506d7544c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:47.412845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-514b0d0c-ed52-4277-88a6-46f506d7544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:47.421538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1306/2589] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:47.426059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69615ee6-26a0-4f84-8f3f-8c647c372e24 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:47.437058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69615ee6-26a0-4f84-8f3f-8c647c372e24 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:47.441798 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:47.442416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.442781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.443190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.443907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:48:47.444220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:47.445936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:47.459185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1284/2590] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:48:47.463835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b0d0c-ed52-4277-88a6-46f506d7544c None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:47.473504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b0d0c-ed52-4277-88a6-46f506d7544c None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:47.479980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:47.480878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.481273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.481650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.482666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:47.485080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:47.573941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1307/2591] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:47 2022] POST /identity/v3/projects => generated 407 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:47.577479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:47.587545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323f63a-02ed-4b0f-9edd-54832f49df3f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:47.644516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:47.645184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.645473 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.645758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.646988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:47.648989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:47.732266 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1285/2592] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:47 2022] POST /identity/v3/users => generated 304 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:47.736449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:47.746974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0a965-d866-41cf-85e7-2d692550c233 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:47.751740 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:47.752355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.752668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.752941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.753637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:47.755225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:47.767954 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1308/2593] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:48:47.774151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:47.784570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3a074-e123-44a5-b67a-02e50d3423c3 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:47.790059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:47.790667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.790983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.791262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.791963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:47.793593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:47.804906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1286/2594] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:47.809719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:47.819781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e2433-769f-445d-80f0-3c47647e4bd5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:47.824311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc6332-93e5-4524-a45e-7c003d2a7191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:47.824999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6332-93e5-4524-a45e-7c003d2a7191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:47.825277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6332-93e5-4524-a45e-7c003d2a7191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:47.825552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc6332-93e5-4524-a45e-7c003d2a7191 admin admin] PATH_INFO: `/v3/projects/ee60b22ac7164d18946ef3ff17055a8b/users/d3461f176586429294d542941664e966/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:47.958116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:47.960402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:47.965254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6332-93e5-4524-a45e-7c003d2a7191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee60b22ac7164d18946ef3ff17055a8b, user_id=d3461f176586429294d542941664e966, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:47.968027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc6332-93e5-4524-a45e-7c003d2a7191 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.004953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:48.018232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a620e-23e8-4f4d-91fa-071b94c442d8 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.022435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.024673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.024673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.024673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.025449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:48.037993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.039438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.046121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1309/2595] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:47 2022] PUT /identity/v3/projects/ee60b22ac7164d18946ef3ff17055a8b/users/d3461f176586429294d542941664e966/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:48.049105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1287/2596] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:48.062843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cc1fb-3dd8-4f59-97a3-6cb4b3d35c59 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.063135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cc1fb-3dd8-4f59-97a3-6cb4b3d35c59 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.063552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cc1fb-3dd8-4f59-97a3-6cb4b3d35c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.091990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f14cc1fb-3dd8-4f59-97a3-6cb4b3d35c59 None None] MFA Rules not processed for user `d3461f176586429294d542941664e966`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:48.093321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f14cc1fb-3dd8-4f59-97a3-6cb4b3d35c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:48.158198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1310/2597] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:48.163592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.172758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac311ab-62bd-475f-b889-f174faa889cc service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.231663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.232283 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.232592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.232865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.233534 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.235446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.317744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1288/2598] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:48 2022] POST /identity/v3/projects => generated 409 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:48.323552 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cc1fb-3dd8-4f59-97a3-6cb4b3d35c59 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.334033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cc1fb-3dd8-4f59-97a3-6cb4b3d35c59 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.401403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.402124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.402460 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.402790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.404114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.405723 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.495927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1311/2599] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:48 2022] POST /identity/v3/users => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:48.505119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.516941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e286c-79ef-4186-af80-50af4dbdaf89 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.522830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.523443 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.523721 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.524023 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.524857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.526437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.537877 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1289/2600] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:48.543113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.553788 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4b8c6-c90d-4ce9-9950-3d9220135fac admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.558718 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.559670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.559670 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.559927 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] PATH_INFO: `/v3/projects/115dfdb6b2dc4fe99ecbca81a5ce0e9c/users/c2a0c45e58234c7eb4ed789ea7420f72/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.562307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:INSERT=6,keystone:DELETE=1,keystone:SELECT=93 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:48.575458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115dfdb6b2dc4fe99ecbca81a5ce0e9c, user_id=c2a0c45e58234c7eb4ed789ea7420f72, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.578128 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.597033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.598946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:48.652908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1312/2601] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:48 2022] PUT /identity/v3/projects/115dfdb6b2dc4fe99ecbca81a5ce0e9c/users/c2a0c45e58234c7eb4ed789ea7420f72/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:48.657001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.669665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2532aa-9694-4587-9e68-5d6be53902f2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.729485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcf218-87be-40e0-8869-47a4956449a2 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.731531 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:48.735209 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.735831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.741045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.741045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.741045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.741045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.741665 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.742233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.742516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.742791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.744683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:48.752313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1313/2602] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:48:48.752313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.753630 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:48.760966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.762942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.767305 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec8448-b7f8-4e41-a75f-6bc6e16eb477 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.772659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.773302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.773620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.773856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] PATH_INFO: `/v3/users/8f2d8cdf0e0d48f4a89a0e85694c3372` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.774478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1290/2603] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:48:48.776268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.784159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2d8cdf0e0d48f4a89a0e85694c3372)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.786159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.787869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0f8a2-e485-48ac-93f8-9d49b587a093 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:48.794236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f122c-2778-471d-a405-67f02c7ecbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:48.794915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f122c-2778-471d-a405-67f02c7ecbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.795191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f122c-2778-471d-a405-67f02c7ecbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.795468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f122c-2778-471d-a405-67f02c7ecbd0 admin admin] PATH_INFO: `/v3/projects/115dfdb6b2dc4fe99ecbca81a5ce0e9c/users/c2a0c45e58234c7eb4ed789ea7420f72/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.798205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f122c-2778-471d-a405-67f02c7ecbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115dfdb6b2dc4fe99ecbca81a5ce0e9c, user_id=c2a0c45e58234c7eb4ed789ea7420f72, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:48.800882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f122c-2778-471d-a405-67f02c7ecbd0 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:48.813782 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1291/2604] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:48 2022] PUT /identity/v3/projects/115dfdb6b2dc4fe99ecbca81a5ce0e9c/users/c2a0c45e58234c7eb4ed789ea7420f72/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:48.824673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:48.824958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:48.825253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:48.862479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] MFA Rules not processed for user `c2a0c45e58234c7eb4ed789ea7420f72`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:48.863963 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:48.936978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1292/2605] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:48 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:48:48.949384 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:48.950807 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:48.978007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2d8cdf0e0d48f4a89a0e85694c3372'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:48.979018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2d8cdf0e0d48f4a89a0e85694c3372'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.012857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7f77-692d-4b14-af84-93cc3cdca789 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:49.016374 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:49.017066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:49.017323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:49.017593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:49.027333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:49.028784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:49.038350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1293/2606] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:48:49.040128 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:49.040573 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:49.064054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2d8cdf0e0d48f4a89a0e85694c3372'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.087257 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2d8cdf0e0d48f4a89a0e85694c3372'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.089114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2d8cdf0e0d48f4a89a0e85694c3372'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.119204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1314/2607] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:48 2022] DELETE /identity/v3/users/8f2d8cdf0e0d48f4a89a0e85694c3372 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:49.145263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:49.154044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd391f8-5369-493c-9c16-c08e6ff57cba service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:49.222309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:49.222951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:49.223232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:49.223499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:49.225790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:49.238218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:49.240920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:49.252193 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1294/2608] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:49.396675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:49.398149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:49.463124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:49.464771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:49.466905 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:49.480424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c3612-c577-4c03-8b99-53f5f879fde8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:49.485216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:49.485859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:49.486138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:49.487002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] PATH_INFO: `/v3/projects/8054a20f70e6408ebe06c787887d0b16` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:49.488184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8054a20f70e6408ebe06c787887d0b16)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:49.491090 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:49.533926 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809f9b0-2135-4aaf-9adf-caa9837de91b service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:49.538313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:49.539100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:49.539448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:49.539789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:49.541718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:49.604251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:49.606390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:49.616629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8054a20f70e6408ebe06c787887d0b16', 'request_id': 'req-97b9253f-98bb-4f8f-a257-f4d720964c0e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.619270 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1295/2609] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:48:49.635011 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8054a20f70e6408ebe06c787887d0b16', 'request_id': 'req-97b9253f-98bb-4f8f-a257-f4d720964c0e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.642771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8054a20f70e6408ebe06c787887d0b16', 'request_id': 'req-97b9253f-98bb-4f8f-a257-f4d720964c0e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.766679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] The token cache is being invalidate because project 8054a20f70e6408ebe06c787887d0b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:49.766969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.767580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:49.769129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1315/2610] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:49 2022] DELETE /identity/v3/projects/8054a20f70e6408ebe06c787887d0b16 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:53.146308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:53.147778 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:53.163827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14562fcb-88c9-4402-893c-8d8f01a932c4 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:53.205019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:53.229149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7841a-d8e3-4ac6-8b63-13f468dd8622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:53.229767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7841a-d8e3-4ac6-8b63-13f468dd8622 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:53.230052 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7841a-d8e3-4ac6-8b63-13f468dd8622 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:53.230315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7841a-d8e3-4ac6-8b63-13f468dd8622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:53.232085 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bbd7841a-d8e3-4ac6-8b63-13f468dd8622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:53.246244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7841a-d8e3-4ac6-8b63-13f468dd8622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:53.247765 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7841a-d8e3-4ac6-8b63-13f468dd8622 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:53.259109 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1296/2611] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:59.328007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:59.329473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:59.395513 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9253f-98bb-4f8f-a257-f4d720964c0e admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:59.400162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:59.400826 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:59.401103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:59.401377 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] PATH_INFO: `/v3/users/d3461f176586429294d542941664e966` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:59.402544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3461f176586429294d542941664e966)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:59.404527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:59.602761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3461f176586429294d542941664e966'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:59.603365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3461f176586429294d542941664e966'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:59.685346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3461f176586429294d542941664e966'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:59.697924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3461f176586429294d542941664e966'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:59.698594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3461f176586429294d542941664e966'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:59.708171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:59.712521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1316/2612] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:59 2022] DELETE /identity/v3/users/d3461f176586429294d542941664e966 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:59.717124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:59.717622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:59.717727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:59.737191 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:59.738424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:59.797402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1297/2613] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:59.807228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:59.808707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:59.866947 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaee251-0bfa-4be0-9125-0b635106652c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:59.870534 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:59.871150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:59.871429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:59.871705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:59.880843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:59.882243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:59.891665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1317/2614] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:00.264514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:00.266019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:00.278988 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eebc2d4-6972-4c48-bee7-8c56d0f55705 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:00.283739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:00.284586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:00.284865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:00.285138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] PATH_INFO: `/v3/projects/ee60b22ac7164d18946ef3ff17055a8b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:00.286320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee60b22ac7164d18946ef3ff17055a8b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:00.288250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:00.396251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee60b22ac7164d18946ef3ff17055a8b', 'request_id': 'req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.402540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee60b22ac7164d18946ef3ff17055a8b', 'request_id': 'req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.408150 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee60b22ac7164d18946ef3ff17055a8b', 'request_id': 'req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.502379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] The token cache is being invalidate because project ee60b22ac7164d18946ef3ff17055a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:49:00.502672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.503320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.505518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1298/2615] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:49:00 2022] DELETE /identity/v3/projects/ee60b22ac7164d18946ef3ff17055a8b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:00.509591 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:00.519265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0da4c-b30a-4c5b-b593-16690ef5e94d service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:00.574472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:00.575102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:00.575382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:00.575659 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] PATH_INFO: `/v3/users/c2a0c45e58234c7eb4ed789ea7420f72` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:00.576981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2a0c45e58234c7eb4ed789ea7420f72)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:00.578890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:00.738287 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2a0c45e58234c7eb4ed789ea7420f72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.739680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2a0c45e58234c7eb4ed789ea7420f72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.805397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2a0c45e58234c7eb4ed789ea7420f72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.817553 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2a0c45e58234c7eb4ed789ea7420f72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.818183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-414bc02b-42de-4aa4-9e0e-0785672c46b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2a0c45e58234c7eb4ed789ea7420f72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:00.830238 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1318/2616] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:49:00 2022] DELETE /identity/v3/users/c2a0c45e58234c7eb4ed789ea7420f72 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:01.229503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:01.230954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:01.244861 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56ee72-8eb2-4340-86a5-d59c486f7aa5 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:01.292062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:01.302020 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:01.302950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:01.304255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:01.304255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] PATH_INFO: `/v3/projects/115dfdb6b2dc4fe99ecbca81a5ce0e9c` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:01.304542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=115dfdb6b2dc4fe99ecbca81a5ce0e9c)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:01.306450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:01.428901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115dfdb6b2dc4fe99ecbca81a5ce0e9c', 'request_id': 'req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:01.435114 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115dfdb6b2dc4fe99ecbca81a5ce0e9c', 'request_id': 'req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:01.440631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115dfdb6b2dc4fe99ecbca81a5ce0e9c', 'request_id': 'req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:01.524999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] The token cache is being invalidate because project 115dfdb6b2dc4fe99ecbca81a5ce0e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:49:01.525286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:01.525956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ed29b720-8bb3-4ced-acae-30ddeb3bfd6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:01.527510 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1299/2617] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:49:01 2022] DELETE /identity/v3/projects/115dfdb6b2dc4fe99ecbca81a5ce0e9c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:01.532364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657953e-3b8f-4c61-b389-2fc4c8baf38a None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:01.532884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657953e-3b8f-4c61-b389-2fc4c8baf38a None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:01.533006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657953e-3b8f-4c61-b389-2fc4c8baf38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:01.553397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c657953e-3b8f-4c61-b389-2fc4c8baf38a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:01.554516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c657953e-3b8f-4c61-b389-2fc4c8baf38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:01.571433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:01.611886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1319/2618] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:01.620487 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307abf-cd4e-43d3-95f9-94b73cc68d99 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:01.620773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307abf-cd4e-43d3-95f9-94b73cc68d99 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:01.621044 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9307abf-cd4e-43d3-95f9-94b73cc68d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:01.640723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d9307abf-cd4e-43d3-95f9-94b73cc68d99 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:01.641914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d9307abf-cd4e-43d3-95f9-94b73cc68d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:01.649723 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1300/2619] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:01.653835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657953e-3b8f-4c61-b389-2fc4c8baf38a None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:01.663787 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657953e-3b8f-4c61-b389-2fc4c8baf38a None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:01.668802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:01.669778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:01.670112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:01.670388 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:01.671085 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:49:01.671392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:01.673016 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:01.685176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1320/2620] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:49:01.689423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9307abf-cd4e-43d3-95f9-94b73cc68d99 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:01.698779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9307abf-cd4e-43d3-95f9-94b73cc68d99 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:01.703468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:01.704099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:01.704399 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:01.704688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:01.705359 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:01.707272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:01.795823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1301/2621] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:49:01 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:49:01.799846 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:01.810427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef083a-0a33-4475-ab51-58931f211f9f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:01.865389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:01.865993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:01.866277 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:01.866555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:01.867228 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:01.869146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:01.971409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1321/2622] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:49:01 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:49:01.975845 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:01.986477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a64d59-53cb-4ce4-9ca3-c7d629ac7bb5 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:01.991424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:01.992028 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:01.992311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:01.992615 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:01.993314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:01.994887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.004804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1302/2623] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:49:02.008626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.018401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6aef14-38c3-4d3a-bfe5-ac244cf852e7 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.023839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db938a3e-03d2-4554-a163-60316b6e7993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.024485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db938a3e-03d2-4554-a163-60316b6e7993 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.024771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db938a3e-03d2-4554-a163-60316b6e7993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.025048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db938a3e-03d2-4554-a163-60316b6e7993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.025747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db938a3e-03d2-4554-a163-60316b6e7993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.028026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db938a3e-03d2-4554-a163-60316b6e7993 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.039326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1322/2624] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:49:02.043218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.053133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64674b5-618b-4b0f-8c50-a2704fd5fe50 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.057719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.058365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.058649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.058931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] PATH_INFO: `/v3/projects/6c1d5d4a8eb045838b093d17de9bbcb3/users/ea221c29de1d4cc8a97f6af5a9070b2b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.073315 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1d5d4a8eb045838b093d17de9bbcb3, user_id=ea221c29de1d4cc8a97f6af5a9070b2b, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.075908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.178364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1303/2625] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:49:02 2022] PUT /identity/v3/projects/6c1d5d4a8eb045838b093d17de9bbcb3/users/ea221c29de1d4cc8a97f6af5a9070b2b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:02.189925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa3fdb-4160-4de3-b827-8f6d51528801 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.190198 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa3fdb-4160-4de3-b827-8f6d51528801 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.190472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa3fdb-4160-4de3-b827-8f6d51528801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.217801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-36fa3fdb-4160-4de3-b827-8f6d51528801 None None] MFA Rules not processed for user `ea221c29de1d4cc8a97f6af5a9070b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:02.218994 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-36fa3fdb-4160-4de3-b827-8f6d51528801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:02.282726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1323/2626] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:02.287736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.349250 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ac55e-9780-49f6-9868-e403f39edae2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.353934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.354549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.354834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.355113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.355795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.357722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.493346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1304/2627] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:49:02 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:49:02.497461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa3fdb-4160-4de3-b827-8f6d51528801 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.508692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa3fdb-4160-4de3-b827-8f6d51528801 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.521360 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:02.564518 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.567540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.567540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.567540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.567540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.568890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.668086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1324/2628] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:49:02 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:49:02.672186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.678483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:02.683803 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3b0c8-238a-47d1-9076-1c076ba41c68 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.688458 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.689067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.689350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.689624 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.690402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.692662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.703133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1305/2629] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:49:02.707240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.717964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c3bc-2097-4454-90e7-ce492b29b962 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.722666 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.723312 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.723586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.723857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] PATH_INFO: `/v3/projects/a816c4b07c934d7b91e978b757e01e10/users/58a735b0dad240b6b503d35ee32fced3/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.738162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a816c4b07c934d7b91e978b757e01e10, user_id=58a735b0dad240b6b503d35ee32fced3, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.741692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.824117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1325/2630] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:49:02 2022] PUT /identity/v3/projects/a816c4b07c934d7b91e978b757e01e10/users/58a735b0dad240b6b503d35ee32fced3/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:02.828287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.838719 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbf088-4761-4238-9c28-a98d3868b981 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.898215 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fb350-1d53-4b8a-a929-d36492628856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.898834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fb350-1d53-4b8a-a929-d36492628856 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.899119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fb350-1d53-4b8a-a929-d36492628856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.899398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fb350-1d53-4b8a-a929-d36492628856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.900117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fb350-1d53-4b8a-a929-d36492628856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.901729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fb350-1d53-4b8a-a929-d36492628856 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.911369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1306/2631] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:49:02.915497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:02.925695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea23662-b835-4411-85f6-793aac3bfaee admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:02.930748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:02.931409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.931692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.932046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] PATH_INFO: `/v3/projects/a816c4b07c934d7b91e978b757e01e10/users/58a735b0dad240b6b503d35ee32fced3/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.934029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a816c4b07c934d7b91e978b757e01e10, user_id=58a735b0dad240b6b503d35ee32fced3, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:02.936642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:02.947108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1326/2632] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:49:02 2022] PUT /identity/v3/projects/a816c4b07c934d7b91e978b757e01e10/users/58a735b0dad240b6b503d35ee32fced3/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:02.957119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15849d-8ffd-4433-a7e0-a94dc3fe53a5 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:02.957404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15849d-8ffd-4433-a7e0-a94dc3fe53a5 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:02.957752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15849d-8ffd-4433-a7e0-a94dc3fe53a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:02.985246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-3c15849d-8ffd-4433-a7e0-a94dc3fe53a5 None None] MFA Rules not processed for user `58a735b0dad240b6b503d35ee32fced3`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:02.986434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3c15849d-8ffd-4433-a7e0-a94dc3fe53a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:03.054259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1307/2633] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:49:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:03.065976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:03.067724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:03.126163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf3d4d-3fde-460e-8e24-24434c923bef admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:03.129849 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc735d-59fe-4b4f-a5be-a694db57d30f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:03.130495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc735d-59fe-4b4f-a5be-a694db57d30f service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:03.130719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc735d-59fe-4b4f-a5be-a694db57d30f service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:03.130991 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc735d-59fe-4b4f-a5be-a694db57d30f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:03.139803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc735d-59fe-4b4f-a5be-a694db57d30f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:03.141256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc735d-59fe-4b4f-a5be-a694db57d30f service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:03.151278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1327/2634] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:49:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:04.197939 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15849d-8ffd-4433-a7e0-a94dc3fe53a5 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:04.258389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15849d-8ffd-4433-a7e0-a94dc3fe53a5 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:04.262865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:04.263486 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:04.263763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:04.264034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] PATH_INFO: `/v3/users/ea221c29de1d4cc8a97f6af5a9070b2b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:04.265209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea221c29de1d4cc8a97f6af5a9070b2b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:04.267103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:04.280119 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:04.442472 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea221c29de1d4cc8a97f6af5a9070b2b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:04.443158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea221c29de1d4cc8a97f6af5a9070b2b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:04.531304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea221c29de1d4cc8a97f6af5a9070b2b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:04.544714 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea221c29de1d4cc8a97f6af5a9070b2b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:04.552408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea221c29de1d4cc8a97f6af5a9070b2b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:04.558449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1308/2635] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:49:04 2022] DELETE /identity/v3/users/ea221c29de1d4cc8a97f6af5a9070b2b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:04.563037 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:04.563520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:04.563626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:04.581696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:04.582890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:04.643858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1328/2636] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:04.653633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:04.655067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:04.711073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f4e7d-1b1f-40b5-ae4a-51b7b0ed33ea admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:04.714675 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:04.715276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:04.715581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:04.715853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:04.725229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:04.726646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:04.735930 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1309/2637] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:05.104222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:05.105657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:05.119142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02fc4-a5ce-4886-a9ba-4edc4257607d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:05.124044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:05.124715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:05.124988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:05.125751 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] PATH_INFO: `/v3/projects/6c1d5d4a8eb045838b093d17de9bbcb3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:05.127156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1d5d4a8eb045838b093d17de9bbcb3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:05.129059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:05.137641 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:05.217442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1d5d4a8eb045838b093d17de9bbcb3', 'request_id': 'req-5ef4353a-635c-439e-94ff-934490bce512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.224073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1d5d4a8eb045838b093d17de9bbcb3', 'request_id': 'req-5ef4353a-635c-439e-94ff-934490bce512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.229667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1d5d4a8eb045838b093d17de9bbcb3', 'request_id': 'req-5ef4353a-635c-439e-94ff-934490bce512'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.301481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] The token cache is being invalidate because project 6c1d5d4a8eb045838b093d17de9bbcb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:49:05.301772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.302392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.303910 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1329/2638] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:49:05 2022] DELETE /identity/v3/projects/6c1d5d4a8eb045838b093d17de9bbcb3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:49:05.307867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:05.318317 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88e84a-f306-47d1-af2a-23948c5c00b4 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:05.376080 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:05.376760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:05.377045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:05.377323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] PATH_INFO: `/v3/users/58a735b0dad240b6b503d35ee32fced3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:05.378491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58a735b0dad240b6b503d35ee32fced3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:05.381182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:05.589276 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:05.597511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58a735b0dad240b6b503d35ee32fced3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.598231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58a735b0dad240b6b503d35ee32fced3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.651154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58a735b0dad240b6b503d35ee32fced3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.664526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58a735b0dad240b6b503d35ee32fced3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.671466 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58a735b0dad240b6b503d35ee32fced3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:05.678069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1310/2639] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:49:05 2022] DELETE /identity/v3/users/58a735b0dad240b6b503d35ee32fced3 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:06.052715 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:06.054192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:06.069423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4353a-635c-439e-94ff-934490bce512 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:06.129059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:06.129804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:06.130091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:06.130370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] PATH_INFO: `/v3/projects/a816c4b07c934d7b91e978b757e01e10` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:06.131537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a816c4b07c934d7b91e978b757e01e10)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:06.133580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:06.222237 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a816c4b07c934d7b91e978b757e01e10', 'request_id': 'req-a649e7ac-4113-4758-800c-0cab0644a95b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:06.228777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a816c4b07c934d7b91e978b757e01e10', 'request_id': 'req-a649e7ac-4113-4758-800c-0cab0644a95b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:06.234219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a816c4b07c934d7b91e978b757e01e10', 'request_id': 'req-a649e7ac-4113-4758-800c-0cab0644a95b'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:06.295586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] The token cache is being invalidate because project a816c4b07c934d7b91e978b757e01e10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:49:06.295866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:06.296541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a649e7ac-4113-4758-800c-0cab0644a95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:06.298027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1330/2640] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:49:06 2022] DELETE /identity/v3/projects/a816c4b07c934d7b91e978b757e01e10 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:15.770924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=15,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:16.292743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:32.418387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:32.423999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:32.523844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c0c58-700d-47ca-9c93-4dae8e84d762 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:32.603454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:32.604169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:32.604650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:32.604978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:32.607372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:32.623855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:32.625501 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:32.637416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1311/2641] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:49:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:42.632631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=62 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:45.721537 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:45.722039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:45.722184 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:45.740983 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] MFA Rules not processed for user `3b94ad3c2863476ea5067d09818f66ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:45.742225 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:45.795445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1331/2642] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:49:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:46.101512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:46.103029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:46.161220 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aec9eb-003a-4a5f-9495-a24699d417f8 service nova] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:46.164983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a8a2b-0f00-4e90-b867-c5878215233d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:46.165547 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a8a2b-0f00-4e90-b867-c5878215233d service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:46.165833 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a8a2b-0f00-4e90-b867-c5878215233d service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:46.166107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a8a2b-0f00-4e90-b867-c5878215233d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:46.167910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f18a8a2b-0f00-4e90-b867-c5878215233d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:46.178004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a8a2b-0f00-4e90-b867-c5878215233d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:46.179416 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a8a2b-0f00-4e90-b867-c5878215233d service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:46.187448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1312/2643] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:49:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:55.788981 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=25 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:56.183788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=26 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:50:00.488963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:50:00.490429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:50:00.506719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32074498-db4c-4969-9702-fc5608fa72bf None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:50:00.512177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:50:00.512837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:50:00.513114 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:50:00.513383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:50:00.515621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:50:00.526310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:50:00.527957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:50:00.537262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1332/2644] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:50:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:50:10.533348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:01.923747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:01.927742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:01.927742 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:01.943792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] MFA Rules not processed for user `3b94ad3c2863476ea5067d09818f66ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:01.945091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:01.952212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1313/2645] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:51:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:02.263604 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:02.273450 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7ce78-f1f0-4c65-bdbf-77ddf2a8de70 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:02.277023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:02.277638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:02.277917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:02.278190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:02.279999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:02.302573 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:02.303987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:02.312646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1333/2646] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:11.938284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:12.308504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=11 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:19.305076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:19.306580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:19.364346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c50e6-d3af-4803-934e-b191fe153e0b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:19.367856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:19.368507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:19.368792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:19.369066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:19.370832 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:19.450726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:19.452763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:19.471188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1314/2647] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:29.468648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=61 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:58.331359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:58.340322 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586fe8-3a75-4db2-826a-8d1987061a6f service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:58.344084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:58.344728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:58.345000 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:58.345268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:58.346997 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:58.356596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:58.357966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:58.366055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1334/2648] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:08.362541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=6 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:21.762117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:21.763557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:21.775709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d57098-0d30-4cdd-9d1a-e74da35fc831 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:21.779323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:21.779924 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:21.780209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:21.780516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:21.782936 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:21.845379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:21.846746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:21.856151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1315/2649] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:25.273512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:25.277335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:25.351853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0d579-7f5e-4a3e-a0f4-4f15090af2fa service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:25.357262 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:25.358042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:25.358250 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:25.359151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:25.362004 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:25.372527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:25.375553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:25.385578 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:25.387428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:25.391552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708c2b0-cbb2-46a8-b54e-6c98dbc30ae7 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:25.397124 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:25.397847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:25.398219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:25.398567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] PATH_INFO: `/v3/users/3b94ad3c2863476ea5067d09818f66ba` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:25.398729 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1335/2650] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:25.400027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b94ad3c2863476ea5067d09818f66ba)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:25.402064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:25.599351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b94ad3c2863476ea5067d09818f66ba'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:25.599969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b94ad3c2863476ea5067d09818f66ba'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:25.655029 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b94ad3c2863476ea5067d09818f66ba'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:25.667371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b94ad3c2863476ea5067d09818f66ba'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:25.674647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-bd2fa638-7048-4f50-afb0-ed78d78cc3b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b94ad3c2863476ea5067d09818f66ba'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:25.680503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1316/2651] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:25 2022] DELETE /identity/v3/users/3b94ad3c2863476ea5067d09818f66ba => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:25.948256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:25.959263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5b599-8345-4611-8875-f22ea9ff27cb service placement] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:26.017243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:26.017881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.018165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.018443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] PATH_INFO: `/v3/projects/345a49c97edc4fb7b1ce7126ce48b5b5` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.019596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=345a49c97edc4fb7b1ce7126ce48b5b5)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:26.021516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:26.141667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345a49c97edc4fb7b1ce7126ce48b5b5', 'request_id': 'req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:26.147851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345a49c97edc4fb7b1ce7126ce48b5b5', 'request_id': 'req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:26.153594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345a49c97edc4fb7b1ce7126ce48b5b5', 'request_id': 'req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:26.220292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] The token cache is being invalidate because project 345a49c97edc4fb7b1ce7126ce48b5b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:26.220551 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:26.221227 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-119f4dd7-eb78-47be-8b2b-4f00a54e0f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:26.222761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1336/2652] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:25 2022] DELETE /identity/v3/projects/345a49c97edc4fb7b1ce7126ce48b5b5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:26.228129 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a641d5-6fe4-4a55-affa-c5deb785bca3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.228650 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a641d5-6fe4-4a55-affa-c5deb785bca3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.228971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a641d5-6fe4-4a55-affa-c5deb785bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.248779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-e8a641d5-6fe4-4a55-affa-c5deb785bca3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:26.249999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-e8a641d5-6fe4-4a55-affa-c5deb785bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:26.308024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1317/2653] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:26.315863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332536f-084f-49f0-96fb-ecac076d0377 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.316410 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332536f-084f-49f0-96fb-ecac076d0377 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.316517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332536f-084f-49f0-96fb-ecac076d0377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.337080 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-1332536f-084f-49f0-96fb-ecac076d0377 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:26.338284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1332536f-084f-49f0-96fb-ecac076d0377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:26.347310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1337/2654] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:26.351464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a641d5-6fe4-4a55-affa-c5deb785bca3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:26.361341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a641d5-6fe4-4a55-affa-c5deb785bca3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:26.365880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:26.366476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.366758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.367027 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.367717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:52:26.368019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:26.369626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:26.371048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:26.381654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1318/2655] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:52:26.385802 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332536f-084f-49f0-96fb-ecac076d0377 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:26.395382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332536f-084f-49f0-96fb-ecac076d0377 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:26.400012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:26.400636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.400917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.401191 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.401904 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:26.403791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:26.415794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:26.490073 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1338/2656] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:26 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:52:26.494198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:26.504614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56bc1b-4873-4759-bdc5-b9cde819857e admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:26.558013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:26.558619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.558900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.559172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.559839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:26.561767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:26.649135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1319/2657] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:26 2022] POST /identity/v3/users => generated 278 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:52:26.653425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:26.664140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f139c-22c6-4040-a5bc-a1a13879a3fb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:26.668939 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:26.669535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.669818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.670094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.673200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:26.674770 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:26.684301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1339/2658] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:26.688117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:26.698072 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74e016-e66c-4c2e-90f2-222f857c4e80 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:26.702670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f652-11d6-4cd5-9b7d-c56e5fe9e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:26.703268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f652-11d6-4cd5-9b7d-c56e5fe9e914 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.703553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f652-11d6-4cd5-9b7d-c56e5fe9e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.703830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f652-11d6-4cd5-9b7d-c56e5fe9e914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.704558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f652-11d6-4cd5-9b7d-c56e5fe9e914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:26.706148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f652-11d6-4cd5-9b7d-c56e5fe9e914 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:26.716820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1320/2659] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:52:26.721175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:26.731134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc32d3-8684-4312-aa3b-5f6fa114b390 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:26.736813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:26.737502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.737781 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.738056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] PATH_INFO: `/v3/projects/d9111c97b6274c16bd93f82f3078f593/users/4e36efa5e0b64cbf8acb6356713c6084/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.752437 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9111c97b6274c16bd93f82f3078f593, user_id=4e36efa5e0b64cbf8acb6356713c6084, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:26.754979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:26.861866 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1340/2660] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:26 2022] PUT /identity/v3/projects/d9111c97b6274c16bd93f82f3078f593/users/4e36efa5e0b64cbf8acb6356713c6084/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:26.927663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d81a4-886f-47b0-870a-3426152bc40a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:26.927951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d81a4-886f-47b0-870a-3426152bc40a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:26.928228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d81a4-886f-47b0-870a-3426152bc40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:26.955990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-731d81a4-886f-47b0-870a-3426152bc40a None None] MFA Rules not processed for user `4e36efa5e0b64cbf8acb6356713c6084`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:26.957389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-731d81a4-886f-47b0-870a-3426152bc40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:27.021171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1321/2661] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:27.025711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:27.085903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9ba58-cc37-4183-b051-dbd0f912e544 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:27.090646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:27.091254 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.091529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.091800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.092494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:27.094365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:27.223170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1341/2662] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:27 2022] POST /identity/v3/projects => generated 357 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:27.227209 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d81a4-886f-47b0-870a-3426152bc40a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:27.238770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d81a4-886f-47b0-870a-3426152bc40a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:27.298438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:27.298984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.299267 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.299542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.300225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:27.302171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:27.310720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:27.409522 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1322/2663] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:27 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:27.413695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:27.424040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e849-1fe5-4a1b-8e9e-5bb877012d75 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:27.428656 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:27.429285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.429564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.429837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.430829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:27.432443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:27.443291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1342/2664] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:27.447491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:27.458205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01065c7-2c72-43ad-bb1f-617991dc3098 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:27.462776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:27.463478 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.463760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.464033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] PATH_INFO: `/v3/projects/405eff7acdee456cafdcdd21d48e5fe8/users/0a9688a411d140aeb25fb27da9849539/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.478612 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] RBAC: Authorizing `identity:create_grant(project_id=405eff7acdee456cafdcdd21d48e5fe8, user_id=0a9688a411d140aeb25fb27da9849539, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:27.481297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:27.551802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1323/2665] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:27 2022] PUT /identity/v3/projects/405eff7acdee456cafdcdd21d48e5fe8/users/0a9688a411d140aeb25fb27da9849539/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:27.555813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:27.566462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f253-fd68-4c90-83dc-38ba7beadc81 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:27.621964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda86853-c592-4cb2-8c66-87398bbb9be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:27.622579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda86853-c592-4cb2-8c66-87398bbb9be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.622908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda86853-c592-4cb2-8c66-87398bbb9be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.623187 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda86853-c592-4cb2-8c66-87398bbb9be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.623880 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda86853-c592-4cb2-8c66-87398bbb9be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:27.625514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda86853-c592-4cb2-8c66-87398bbb9be4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:27.635294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1343/2666] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:27.639385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:27.650204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2534c42-236b-44b8-a379-54fc71b7179a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:27.654900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:27.655569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.655841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.656122 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] PATH_INFO: `/v3/projects/405eff7acdee456cafdcdd21d48e5fe8/users/0a9688a411d140aeb25fb27da9849539/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.658747 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405eff7acdee456cafdcdd21d48e5fe8, user_id=0a9688a411d140aeb25fb27da9849539, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:27.661403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:27.671505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1324/2667] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:27 2022] PUT /identity/v3/projects/405eff7acdee456cafdcdd21d48e5fe8/users/0a9688a411d140aeb25fb27da9849539/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:27.681146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573355-53c8-4bf8-ab37-9ee9ba44b941 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.681438 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573355-53c8-4bf8-ab37-9ee9ba44b941 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.681717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e573355-53c8-4bf8-ab37-9ee9ba44b941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.709123 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-3e573355-53c8-4bf8-ab37-9ee9ba44b941 None None] MFA Rules not processed for user `0a9688a411d140aeb25fb27da9849539`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:27.710326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-3e573355-53c8-4bf8-ab37-9ee9ba44b941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:27.723103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:27.777678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1344/2668] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:27 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:27.789491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:27.791233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:27.850303 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d27776-66ff-47ed-bc7e-80e8ee8280e0 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:27.853901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:27.854783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:27.855071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:27.855346 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:27.865308 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:27.866767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:27.876361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1325/2669] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:27 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:28.061867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e573355-53c8-4bf8-ab37-9ee9ba44b941 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:28.071810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e573355-53c8-4bf8-ab37-9ee9ba44b941 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:28.075339 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:28.075942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:28.076219 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:28.076532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:28.133105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:28.134443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:28.143960 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1345/2670] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:28 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:37.871960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:38.139314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=50 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:43.771795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:43.773313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:43.847646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d6cfa-d55a-4e2e-b494-a6133a011f25 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:43.851299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa165d6-9ab8-4381-be45-0b39d8565262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:43.852265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa165d6-9ab8-4381-be45-0b39d8565262 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:43.852265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa165d6-9ab8-4381-be45-0b39d8565262 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:43.852506 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa165d6-9ab8-4381-be45-0b39d8565262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:43.854229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cfa165d6-9ab8-4381-be45-0b39d8565262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:43.941947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa165d6-9ab8-4381-be45-0b39d8565262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:43.943334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa165d6-9ab8-4381-be45-0b39d8565262 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:43.953070 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1326/2671] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:46.337142 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:46.367389 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eca8e-f69b-4d7c-a437-feda4ac99d19 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:46.372094 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:46.372850 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:46.373131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:46.373411 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] PATH_INFO: `/v3/users/4e36efa5e0b64cbf8acb6356713c6084` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:46.374607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e36efa5e0b64cbf8acb6356713c6084)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:46.376567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:46.543077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e36efa5e0b64cbf8acb6356713c6084'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:46.543933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e36efa5e0b64cbf8acb6356713c6084'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:46.611426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e36efa5e0b64cbf8acb6356713c6084'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:46.620917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e36efa5e0b64cbf8acb6356713c6084'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:46.621602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e36efa5e0b64cbf8acb6356713c6084'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:46.634017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1346/2672] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:46 2022] DELETE /identity/v3/users/4e36efa5e0b64cbf8acb6356713c6084 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:46.640155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:46.640155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:46.640155 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:46.658876 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:46.660364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:46.730987 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1327/2673] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:46.741179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:46.742690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:46.805519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66805cf2-291e-4f72-8026-d801693b665f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:46.809303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:46.809925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:46.810205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:46.810481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:46.821055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:46.822467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:46.833472 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1347/2674] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:47.307311 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:47.309083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:47.322645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab693fa-daa7-4fe1-9aaf-50d118611374 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:47.327424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:47.328062 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:47.328410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:47.328682 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] PATH_INFO: `/v3/projects/d9111c97b6274c16bd93f82f3078f593` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:47.330420 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9111c97b6274c16bd93f82f3078f593)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:47.332579 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:47.341941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:47.468633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9111c97b6274c16bd93f82f3078f593', 'request_id': 'req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.475953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9111c97b6274c16bd93f82f3078f593', 'request_id': 'req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.482415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9111c97b6274c16bd93f82f3078f593', 'request_id': 'req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.573242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] The token cache is being invalidate because project d9111c97b6274c16bd93f82f3078f593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:47.573769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.574526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.576476 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1328/2675] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:47 2022] DELETE /identity/v3/projects/d9111c97b6274c16bd93f82f3078f593 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:47.581040 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:47.592205 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b0bf-4c6a-465a-b5bc-6e67f90f2bcd service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:47.653309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:47.656089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:47.656089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:47.656089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] PATH_INFO: `/v3/users/0a9688a411d140aeb25fb27da9849539` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:47.656089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9688a411d140aeb25fb27da9849539)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:47.657602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:47.659913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:47.684401 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:47.685814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:47.700361 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdeab2-1f53-4795-ab43-d6b8c57a926f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:47.705899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:47.707694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:47.707694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:47.707694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] PATH_INFO: `/v3/users/6cc262a3c3314be8a6ee620ebfa6fe85` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:47.708479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc262a3c3314be8a6ee620ebfa6fe85)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:47.710426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:47.851111 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9688a411d140aeb25fb27da9849539'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.851663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9688a411d140aeb25fb27da9849539'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.896680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc262a3c3314be8a6ee620ebfa6fe85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.897375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc262a3c3314be8a6ee620ebfa6fe85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.911665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9688a411d140aeb25fb27da9849539'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.924364 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9688a411d140aeb25fb27da9849539'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.925023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9688a411d140aeb25fb27da9849539'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.937155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1348/2676] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:47 2022] DELETE /identity/v3/users/0a9688a411d140aeb25fb27da9849539 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:47.963177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc262a3c3314be8a6ee620ebfa6fe85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.976234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc262a3c3314be8a6ee620ebfa6fe85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.984302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc262a3c3314be8a6ee620ebfa6fe85'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:47.991201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1329/2677] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:47 2022] DELETE /identity/v3/users/6cc262a3c3314be8a6ee620ebfa6fe85 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:48.250070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.251525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.317300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0463fae-0acc-4ea0-a2a5-23b090032fdd admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.324951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.326751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.389617 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.390194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.390488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.390754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] PATH_INFO: `/v3/projects/a56bf911ebbf4dd1ac5ed4bb07a9f7b3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.392052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56bf911ebbf4dd1ac5ed4bb07a9f7b3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.394084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.410375 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396801a7-654d-42dd-8d7b-b57553c192e5 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.416587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.417432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.417900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.418271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] PATH_INFO: `/v3/projects/405eff7acdee456cafdcdd21d48e5fe8` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.419827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=405eff7acdee456cafdcdd21d48e5fe8)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.421985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.434969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:48.490102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:48.503580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56bf911ebbf4dd1ac5ed4bb07a9f7b3', 'request_id': 'req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.510199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56bf911ebbf4dd1ac5ed4bb07a9f7b3', 'request_id': 'req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.516173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56bf911ebbf4dd1ac5ed4bb07a9f7b3', 'request_id': 'req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.536915 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405eff7acdee456cafdcdd21d48e5fe8', 'request_id': 'req-098c8410-2c18-4547-a134-883737a7ead8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.542964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405eff7acdee456cafdcdd21d48e5fe8', 'request_id': 'req-098c8410-2c18-4547-a134-883737a7ead8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.548849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405eff7acdee456cafdcdd21d48e5fe8', 'request_id': 'req-098c8410-2c18-4547-a134-883737a7ead8'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.607765 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] The token cache is being invalidate because project a56bf911ebbf4dd1ac5ed4bb07a9f7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:48.608310 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.608952 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dea7fc82-1f72-4d9a-887d-32fe6b2b3dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.610747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1349/2678] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:48 2022] DELETE /identity/v3/projects/a56bf911ebbf4dd1ac5ed4bb07a9f7b3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:48.616299 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf72dd5-a3d8-40ec-84f7-df0034dfe84f None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.616611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf72dd5-a3d8-40ec-84f7-df0034dfe84f None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.616908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf72dd5-a3d8-40ec-84f7-df0034dfe84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.635701 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-fbf72dd5-a3d8-40ec-84f7-df0034dfe84f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:48.636925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fbf72dd5-a3d8-40ec-84f7-df0034dfe84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.682163 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] The token cache is being invalidate because project 405eff7acdee456cafdcdd21d48e5fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:48.682387 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.683045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-098c8410-2c18-4547-a134-883737a7ead8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.684575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1330/2679] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:48 2022] DELETE /identity/v3/projects/405eff7acdee456cafdcdd21d48e5fe8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:48.689200 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3b37b-a851-4a51-9da5-c94bdcfd9b08 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.689526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3b37b-a851-4a51-9da5-c94bdcfd9b08 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.689751 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3b37b-a851-4a51-9da5-c94bdcfd9b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.695112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1350/2680] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:52:48.704662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8992c6-cfad-4d11-84a7-da8d1daf51c2 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.704917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8992c6-cfad-4d11-84a7-da8d1daf51c2 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.705285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8992c6-cfad-4d11-84a7-da8d1daf51c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.711504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-d3d3b37b-a851-4a51-9da5-c94bdcfd9b08 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:48.712768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d3d3b37b-a851-4a51-9da5-c94bdcfd9b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.721385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1331/2681] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:52:48.724942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-9e8992c6-cfad-4d11-84a7-da8d1daf51c2 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:48.726141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9e8992c6-cfad-4d11-84a7-da8d1daf51c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.730397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbe38-fe50-4ae4-9b12-4a6362ac4381 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.730690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbe38-fe50-4ae4-9b12-4a6362ac4381 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.730970 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbbe38-fe50-4ae4-9b12-4a6362ac4381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.736099 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1351/2682] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:52:48.745444 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8992c6-cfad-4d11-84a7-da8d1daf51c2 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.749459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8992c6-cfad-4d11-84a7-da8d1daf51c2 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.751292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-97dbbe38-fe50-4ae4-9b12-4a6362ac4381 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:48.752537 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-97dbbe38-fe50-4ae4-9b12-4a6362ac4381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.754154 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.754755 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.755032 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.755304 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.756019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:52:48.756328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.759210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.760677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1332/2683] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:52:48.764690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbbe38-fe50-4ae4-9b12-4a6362ac4381 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.771579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1352/2684] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:52:48.774125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbbe38-fe50-4ae4-9b12-4a6362ac4381 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.775548 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.778912 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.779511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.779800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.780077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.780812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:52:48.781112 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.783326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.785804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fc757-6944-46b3-aaa5-3b1be8de1eba admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.790315 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.790898 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.791173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.791442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.792105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.793987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.795015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1333/2685] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:52:48.799607 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.810095 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649b5cc-3134-42b3-8ef6-9ebdc5b4cfb7 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.814720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.815301 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.815577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.815849 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.816529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.818364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.861499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1353/2686] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:48.865557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.878817 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1334/2687] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:52:48.882911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.934173 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c37f2-72d3-4c61-9b26-350e3a92cc23 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.955956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:48.969370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288ee85-4cd0-4a11-a450-70578d514419 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.977307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.978178 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.978316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.978556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.979565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.981515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.996566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.997186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.997481 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.997733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.998405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.000279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.072307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1354/2688] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/users => generated 284 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:52:49.076024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1335/2689] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:52:49.079653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.080138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.091211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d85fa-1eaf-44cd-856c-8108def2dab5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.092957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3dee9-aacc-42a9-81bd-68fd629e380a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.096072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.096879 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.097256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.097646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.098427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.098759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.099372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.099652 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.099925 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.100640 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.100763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.102223 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.111588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1355/2690] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:49.113126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1336/2691] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:49.115478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.119228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.125259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88f8e-4a99-484a-ad2c-5ed1cbb4b6ab admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.129418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bab58-e15c-462b-a062-a9aa1983b2b6 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.130623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.131233 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.131512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.131786 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.132881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.134138 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.135219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.136151 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.136669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.137157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.138357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.140570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.146382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1356/2692] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:52:49.150445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.157792 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1337/2693] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:52:49.158875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.160131 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5460dc-d0de-4822-9147-ab0cd540e044 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.168632 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcedcb-9971-486c-9b8d-55065f3333fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.168892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafaa37d-5da5-4acf-9f3c-fdf08d6ff8b6 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.169284 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcedcb-9971-486c-9b8d-55065f3333fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.169566 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcedcb-9971-486c-9b8d-55065f3333fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.169842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcedcb-9971-486c-9b8d-55065f3333fa admin admin] PATH_INFO: `/v3/projects/f44f0949a109409fbcd228d26e0ae6ab/users/a500aa7cef77473c83bcb282a47d2019/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.174802 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad995a13-fa86-4ee2-8937-48577c08a493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.175454 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad995a13-fa86-4ee2-8937-48577c08a493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.175732 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad995a13-fa86-4ee2-8937-48577c08a493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.176003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad995a13-fa86-4ee2-8937-48577c08a493 admin admin] PATH_INFO: `/v3/projects/bca15d7b85704cfd93cbe9e498308632/users/5e07be6c143d43529ebab38bac5b4ff1/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.185445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcedcb-9971-486c-9b8d-55065f3333fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44f0949a109409fbcd228d26e0ae6ab, user_id=a500aa7cef77473c83bcb282a47d2019, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.188051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcedcb-9971-486c-9b8d-55065f3333fa admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.191381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad995a13-fa86-4ee2-8937-48577c08a493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca15d7b85704cfd93cbe9e498308632, user_id=5e07be6c143d43529ebab38bac5b4ff1, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.194005 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad995a13-fa86-4ee2-8937-48577c08a493 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.201882 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:49.265264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1357/2694] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:49 2022] PUT /identity/v3/projects/f44f0949a109409fbcd228d26e0ae6ab/users/a500aa7cef77473c83bcb282a47d2019/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:49.276549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.276837 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.277113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.278314 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1338/2695] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:49 2022] PUT /identity/v3/projects/bca15d7b85704cfd93cbe9e498308632/users/5e07be6c143d43529ebab38bac5b4ff1/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:49.288787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b9b54-fb20-4242-b6c3-7479df9c2a9a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.289079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b9b54-fb20-4242-b6c3-7479df9c2a9a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.289489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b9b54-fb20-4242-b6c3-7479df9c2a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.306174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] MFA Rules not processed for user `a500aa7cef77473c83bcb282a47d2019`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:49.307394 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.317727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ff9b9b54-fb20-4242-b6c3-7479df9c2a9a None None] MFA Rules not processed for user `5e07be6c143d43529ebab38bac5b4ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:49.318916 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ff9b9b54-fb20-4242-b6c3-7479df9c2a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.375504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1358/2696] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:52:49.383409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1339/2697] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:52:49.387217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.387576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b9b54-fb20-4242-b6c3-7479df9c2a9a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.388631 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.447484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545728e3-ba83-49cd-b9df-4012e6281cda None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.450662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b9b54-fb20-4242-b6c3-7479df9c2a9a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.451193 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.451809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.452091 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.452363 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.455347 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.456030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.456312 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.456619 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.457287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.459227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.460773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.462185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.466742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:49.471369 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1359/2698] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:49.534418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1340/2699] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:52:49.535596 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.546708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cd615-efed-4abd-89d0-43e71765f1ce service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.608316 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.609018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.609232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.609763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.610946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.612895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.688355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1360/2700] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:49.692958 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.703490 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0cda-7ad9-44c8-a9f7-a5697b21cbb9 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.708202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.713797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.713797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.713797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.713797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.713797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.722271 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1341/2701] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:49.726349 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.737263 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0f6dc-0449-4617-b759-83f59b798203 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.741815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.742469 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.742780 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.743055 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] PATH_INFO: `/v3/projects/cc88aff7dbe241a3a7342d9b6c75c975/users/6ac747f42f314e22a68046f85d8db0ac/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.757820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc88aff7dbe241a3a7342d9b6c75c975, user_id=6ac747f42f314e22a68046f85d8db0ac, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.760633 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.843137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1361/2702] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:49 2022] PUT /identity/v3/projects/cc88aff7dbe241a3a7342d9b6c75c975/users/6ac747f42f314e22a68046f85d8db0ac/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:49.847598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.857811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf7983-1a8d-46ff-86c8-11a45a4c86fe admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.911956 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:49.917978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fb50c-4817-40e7-968e-9e1b5bd86203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.918598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fb50c-4817-40e7-968e-9e1b5bd86203 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.918884 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fb50c-4817-40e7-968e-9e1b5bd86203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.919160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fb50c-4817-40e7-968e-9e1b5bd86203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.919878 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fb50c-4817-40e7-968e-9e1b5bd86203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.921507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fb50c-4817-40e7-968e-9e1b5bd86203 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.931749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1342/2703] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:49.935987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.946585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff66188-fcb8-459f-9985-b74943e7cc0b admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.951274 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.951933 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.952245 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.952557 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] PATH_INFO: `/v3/projects/cc88aff7dbe241a3a7342d9b6c75c975/users/6ac747f42f314e22a68046f85d8db0ac/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.954598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc88aff7dbe241a3a7342d9b6c75c975, user_id=6ac747f42f314e22a68046f85d8db0ac, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.957192 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.968013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1362/2704] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:49 2022] PUT /identity/v3/projects/cc88aff7dbe241a3a7342d9b6c75c975/users/6ac747f42f314e22a68046f85d8db0ac/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:49.978945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01aebe-bccb-4127-8785-cac0aa090a50 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.979233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01aebe-bccb-4127-8785-cac0aa090a50 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.979504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01aebe-bccb-4127-8785-cac0aa090a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:50.007526 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-dd01aebe-bccb-4127-8785-cac0aa090a50 None None] MFA Rules not processed for user `6ac747f42f314e22a68046f85d8db0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:50.008756 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-dd01aebe-bccb-4127-8785-cac0aa090a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:50.084666 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1343/2705] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:50.097493 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:50.154661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec468-46c6-4c20-a7a1-80518e3b2243 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:50.159012 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:50.159635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:50.159907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:50.160174 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:50.169899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:50.171268 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:50.180771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1363/2706] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:50.440608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01aebe-bccb-4127-8785-cac0aa090a50 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:50.450182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01aebe-bccb-4127-8785-cac0aa090a50 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:50.454463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4d86b-8965-472d-a5e0-a3c2f5f3ee43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:50.455094 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4d86b-8965-472d-a5e0-a3c2f5f3ee43 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:50.455376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4d86b-8965-472d-a5e0-a3c2f5f3ee43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:50.455653 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4d86b-8965-472d-a5e0-a3c2f5f3ee43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:50.511948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4d86b-8965-472d-a5e0-a3c2f5f3ee43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:50.513408 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4d86b-8965-472d-a5e0-a3c2f5f3ee43 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:50.523009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1344/2707] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:56.582382 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:56.640132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:56.645754 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152d00e-105c-4750-b67c-5c91611aa066 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:56.653689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:56.653689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:56.653689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:56.653689 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] PATH_INFO: `/v3/users/5e07be6c143d43529ebab38bac5b4ff1` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:56.654095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e07be6c143d43529ebab38bac5b4ff1)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:56.654747 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:56.797528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e07be6c143d43529ebab38bac5b4ff1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:56.798775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e07be6c143d43529ebab38bac5b4ff1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:56.876897 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e07be6c143d43529ebab38bac5b4ff1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:56.889986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e07be6c143d43529ebab38bac5b4ff1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:56.890657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e07be6c143d43529ebab38bac5b4ff1'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:56.911853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1364/2708] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:56 2022] DELETE /identity/v3/users/5e07be6c143d43529ebab38bac5b4ff1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:56.916259 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:56.919991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:56.919991 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:56.935844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:56.937079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:56.995277 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1345/2709] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:57.005127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:57.006616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:57.062878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8401f-4175-4bba-bdd4-2d4de1e8d3e0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:57.066724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:57.067345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:57.067629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:57.067907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:57.076911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:57.078330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:57.088724 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1365/2710] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:57.495789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:57.498538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:57.510117 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad232a4e-c695-49b7-a67d-1dc15184d00a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:57.514806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:57.515450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:57.515727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:57.515992 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] PATH_INFO: `/v3/projects/bca15d7b85704cfd93cbe9e498308632` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:57.517900 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca15d7b85704cfd93cbe9e498308632)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:57.519935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:57.531918 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:57.639614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca15d7b85704cfd93cbe9e498308632', 'request_id': 'req-18cbf7d8-f791-430a-b49d-ed5d8d601048'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:57.646014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca15d7b85704cfd93cbe9e498308632', 'request_id': 'req-18cbf7d8-f791-430a-b49d-ed5d8d601048'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:57.651554 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca15d7b85704cfd93cbe9e498308632', 'request_id': 'req-18cbf7d8-f791-430a-b49d-ed5d8d601048'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:57.698898 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] The token cache is being invalidate because project bca15d7b85704cfd93cbe9e498308632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:57.699187 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:57.699829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:57.701600 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1346/2711] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:57 2022] DELETE /identity/v3/projects/bca15d7b85704cfd93cbe9e498308632 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:57.705462 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:57.714963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f553241-ce50-4682-ba5c-f83a104b1358 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:57.769241 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:57.769867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:57.770148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:57.770424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] PATH_INFO: `/v3/users/6ac747f42f314e22a68046f85d8db0ac` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:57.771570 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac747f42f314e22a68046f85d8db0ac)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:57.773495 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:57.793143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:57.942434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac747f42f314e22a68046f85d8db0ac'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:57.943122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac747f42f314e22a68046f85d8db0ac'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.002459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac747f42f314e22a68046f85d8db0ac'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.014957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac747f42f314e22a68046f85d8db0ac'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.015602 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-dafc79a2-9f0d-46b2-8491-bbd078238678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac747f42f314e22a68046f85d8db0ac'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.028705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1366/2712] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:57 2022] DELETE /identity/v3/users/6ac747f42f314e22a68046f85d8db0ac => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:52:58.410256 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:58.411836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:58.425435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cbf7d8-f791-430a-b49d-ed5d8d601048 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:58.484577 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:58.485212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:58.485487 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:58.485759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] PATH_INFO: `/v3/projects/cc88aff7dbe241a3a7342d9b6c75c975` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:58.486905 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc88aff7dbe241a3a7342d9b6c75c975)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:58.488822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:58.596927 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc88aff7dbe241a3a7342d9b6c75c975', 'request_id': 'req-ec898683-63d5-4fc4-add0-67b4dd607c81'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.603108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc88aff7dbe241a3a7342d9b6c75c975', 'request_id': 'req-ec898683-63d5-4fc4-add0-67b4dd607c81'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.608594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc88aff7dbe241a3a7342d9b6c75c975', 'request_id': 'req-ec898683-63d5-4fc4-add0-67b4dd607c81'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.681138 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] The token cache is being invalidate because project cc88aff7dbe241a3a7342d9b6c75c975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:58.681431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.682049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ec898683-63d5-4fc4-add0-67b4dd607c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:58.683558 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1347/2713] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:58 2022] DELETE /identity/v3/projects/cc88aff7dbe241a3a7342d9b6c75c975 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:58.688514 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37eed-cc00-49f5-b2d5-011c4cc58247 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:58.689337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37eed-cc00-49f5-b2d5-011c4cc58247 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:58.689564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37eed-cc00-49f5-b2d5-011c4cc58247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:58.710510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-16f37eed-cc00-49f5-b2d5-011c4cc58247 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:58.714197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-16f37eed-cc00-49f5-b2d5-011c4cc58247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:58.769002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1367/2714] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:58.776941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67491c1-d0a3-4ce5-9a15-5c212a7624fc None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:58.777229 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67491c1-d0a3-4ce5-9a15-5c212a7624fc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:58.777503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67491c1-d0a3-4ce5-9a15-5c212a7624fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:58.796768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a67491c1-d0a3-4ce5-9a15-5c212a7624fc None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:58.800769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a67491c1-d0a3-4ce5-9a15-5c212a7624fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:58.806427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1348/2715] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:58.810635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37eed-cc00-49f5-b2d5-011c4cc58247 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:58.820840 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f37eed-cc00-49f5-b2d5-011c4cc58247 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:58.825405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:58.829882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:58.829882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:58.829882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:58.829882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:52:58.829882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:58.829882 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:58.840974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1368/2716] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:52:58.845184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67491c1-d0a3-4ce5-9a15-5c212a7624fc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:58.854658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67491c1-d0a3-4ce5-9a15-5c212a7624fc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:58.859813 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:58.860455 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:58.860738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:58.861013 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:58.861678 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:58.863570 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:58.963949 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1349/2717] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:58 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:58.967995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:58.977995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f6d1b-87f0-42c3-b722-6e4cc922bf0c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:59.033352 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:59.034045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.034323 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.034594 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.035243 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:59.037144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:59.099140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1369/2718] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:58 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:59.103225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:59.108702 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:59.115243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a4da-274f-4d88-97eb-bb60435785f5 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:59.119895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:59.120543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.120824 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.121096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.121795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:59.123372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:59.133989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1350/2719] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:59.138318 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:59.148490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933c9c2-35c6-451f-94de-ae886e749502 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:59.153189 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:59.153816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.154095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.154370 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.155116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:59.157232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:59.173671 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1370/2720] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:59.173794 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:59.178461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:59.180785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:59.182594 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23facd-b7d1-434b-8e0a-e13403bed253 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:59.187251 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a80f39-ac48-439f-822a-a6c979a4b537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:59.187911 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80f39-ac48-439f-822a-a6c979a4b537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.188195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80f39-ac48-439f-822a-a6c979a4b537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.188498 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80f39-ac48-439f-822a-a6c979a4b537 admin admin] PATH_INFO: `/v3/projects/ab2db90ccbbf468587d34d323c7bcd8a/users/bffda1c5c6644b98a4b46c45d5e80a73/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.189258 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:59.194608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16077b2-ee35-43ab-974a-89378365992f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:59.199266 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:59.199892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.200179 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.200496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] PATH_INFO: `/v3/users/a500aa7cef77473c83bcb282a47d2019` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.201683 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a500aa7cef77473c83bcb282a47d2019)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:59.203575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:59.204025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a80f39-ac48-439f-822a-a6c979a4b537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2db90ccbbf468587d34d323c7bcd8a, user_id=bffda1c5c6644b98a4b46c45d5e80a73, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:59.206827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a80f39-ac48-439f-822a-a6c979a4b537 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:59.299741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1351/2721] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:59 2022] PUT /identity/v3/projects/ab2db90ccbbf468587d34d323c7bcd8a/users/bffda1c5c6644b98a4b46c45d5e80a73/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:59.304796 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.305086 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.305363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.324244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:59.325492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:59.383921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1352/2722] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:52:59.393419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:59.394836 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:59.405044 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a500aa7cef77473c83bcb282a47d2019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:59.405749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a500aa7cef77473c83bcb282a47d2019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:59.453100 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd8f0-f1f2-4d84-a098-ddf10bf363cd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:59.456847 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fb37a-3531-4eb7-8b4a-d49552647ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:59.457459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fb37a-3531-4eb7-8b4a-d49552647ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.457759 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fb37a-3531-4eb7-8b4a-d49552647ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.460844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fb37a-3531-4eb7-8b4a-d49552647ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.467300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fb37a-3531-4eb7-8b4a-d49552647ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:59.468757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fb37a-3531-4eb7-8b4a-d49552647ef6 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:59.478679 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1353/2723] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:52:59.499432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a500aa7cef77473c83bcb282a47d2019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:59.511697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a500aa7cef77473c83bcb282a47d2019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:59.515140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a500aa7cef77473c83bcb282a47d2019'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:59.524426 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1371/2724] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:59 2022] DELETE /identity/v3/users/a500aa7cef77473c83bcb282a47d2019 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:59.528966 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.529254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.529529 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.550365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:59.551512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:59.611118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1354/2725] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:59.621510 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:59.623082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:59.684084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be26d9d-ce8c-4e15-9365-09988c4cd35c admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:59.687797 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df9d90-2003-4507-a511-0cde9ede5488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:59.688436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9d90-2003-4507-a511-0cde9ede5488 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:59.688767 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9d90-2003-4507-a511-0cde9ede5488 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:59.689043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9d90-2003-4507-a511-0cde9ede5488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:59.697885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9d90-2003-4507-a511-0cde9ede5488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:59.699300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9d90-2003-4507-a511-0cde9ede5488 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:59.708697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1372/2726] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:00.124265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.125714 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:00.138780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fab9d-8fcd-4861-bf27-02a567a4d36f None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.144230 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.145051 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.145683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.146031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] PATH_INFO: `/v3/projects/f44f0949a109409fbcd228d26e0ae6ab` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.147381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44f0949a109409fbcd228d26e0ae6ab)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.149744 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.158883 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:00.237434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44f0949a109409fbcd228d26e0ae6ab', 'request_id': 'req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:00.243735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44f0949a109409fbcd228d26e0ae6ab', 'request_id': 'req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:00.249285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44f0949a109409fbcd228d26e0ae6ab', 'request_id': 'req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:00.299954 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] The token cache is being invalidate because project f44f0949a109409fbcd228d26e0ae6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:00.300334 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:00.301447 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-67c4aa9c-f29e-4872-b4fb-b3673bc956c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:00.303002 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1355/2727] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:00 2022] DELETE /identity/v3/projects/f44f0949a109409fbcd228d26e0ae6ab => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:00.307968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82103aba-ff0c-4265-bc66-679444a65b17 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.308509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82103aba-ff0c-4265-bc66-679444a65b17 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.308627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82103aba-ff0c-4265-bc66-679444a65b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.328897 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-82103aba-ff0c-4265-bc66-679444a65b17 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:00.330713 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-82103aba-ff0c-4265-bc66-679444a65b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:00.389523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1373/2728] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:00.398299 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5db33-482a-4693-bf31-8ea61c003509 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.398593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5db33-482a-4693-bf31-8ea61c003509 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.398873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5db33-482a-4693-bf31-8ea61c003509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.419773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-34a5db33-482a-4693-bf31-8ea61c003509 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:00.421019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-34a5db33-482a-4693-bf31-8ea61c003509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:00.429914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1356/2729] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:00.434067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82103aba-ff0c-4265-bc66-679444a65b17 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.444178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82103aba-ff0c-4265-bc66-679444a65b17 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.448967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.449572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.449852 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.450126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.450855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:53:00.451162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.452790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.469116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1374/2730] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:53:00.470397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5db33-482a-4693-bf31-8ea61c003509 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.480079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5db33-482a-4693-bf31-8ea61c003509 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.485471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.486517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.486517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.486746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.487410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.489376 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.638118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1357/2731] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:00 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:00.642201 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.648697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:00.652430 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ea631-0ea6-4816-a2cd-a2c280203e3d admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.696917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.698718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:00.716470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dce41e-4534-4a29-993a-bc9c26ba4b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.717235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dce41e-4534-4a29-993a-bc9c26ba4b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.717636 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dce41e-4534-4a29-993a-bc9c26ba4b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.717989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dce41e-4534-4a29-993a-bc9c26ba4b6d admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.718804 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dce41e-4534-4a29-993a-bc9c26ba4b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.720745 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dce41e-4534-4a29-993a-bc9c26ba4b6d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.792217 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1375/2732] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:00 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:00.793646 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd27e1-2916-4c5b-83cd-28470c355aaa None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.794071 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd27e1-2916-4c5b-83cd-28470c355aaa None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.794484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd27e1-2916-4c5b-83cd-28470c355aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.801248 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac5b29-4d5b-4abb-847d-dfff72378a0a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.807527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.808161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.808603 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.808819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.810741 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:00.824133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.825766 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.828419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-9ffd27e1-2916-4c5b-83cd-28470c355aaa None None] MFA Rules not processed for user `bffda1c5c6644b98a4b46c45d5e80a73`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:00.829860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9ffd27e1-2916-4c5b-83cd-28470c355aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:00.836186 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1358/2733] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:00.838774 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.849159 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255b4e3-5788-42b4-8b58-f331f2d2e289 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.854040 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.854663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.854951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.855232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.855940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.857575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.871108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1359/2734] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:00.877844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.888077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c7001-293b-47e8-9887-edbe0c82b3bc admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.892971 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.893528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.893814 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.894524 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.895323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.901030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.911075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1376/2735] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:00.924852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1360/2736] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:53:00.925466 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd27e1-2916-4c5b-83cd-28470c355aaa None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.925589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:00.925589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:00.929473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd27e1-2916-4c5b-83cd-28470c355aaa None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.934178 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.934848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.935149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.935470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] PATH_INFO: `/v3/projects/5bda1b7e36df4f82b16176476faa60b7/users/e0afcee881974443a053c0a5eaad3314/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:00.940829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:00.952074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bda1b7e36df4f82b16176476faa60b7, user_id=e0afcee881974443a053c0a5eaad3314, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:00.954772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:00.987400 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323637-ff42-4358-9ec3-1af91870a857 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:00.991252 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a8b94-db7e-4147-8f76-6a8df7ecb7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:00.991870 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a8b94-db7e-4147-8f76-6a8df7ecb7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:00.992154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a8b94-db7e-4147-8f76-6a8df7ecb7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:00.992464 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a8b94-db7e-4147-8f76-6a8df7ecb7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.000868 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a8b94-db7e-4147-8f76-6a8df7ecb7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:01.002285 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a8b94-db7e-4147-8f76-6a8df7ecb7a1 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:01.011351 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1361/2737] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:53:01.070987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1377/2738] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:00 2022] PUT /identity/v3/projects/5bda1b7e36df4f82b16176476faa60b7/users/e0afcee881974443a053c0a5eaad3314/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:01.143551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5434-76a3-4f92-a6f6-09e29b7fd29b None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.143875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5434-76a3-4f92-a6f6-09e29b7fd29b None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.144125 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e5434-76a3-4f92-a6f6-09e29b7fd29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.174822 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f80e5434-76a3-4f92-a6f6-09e29b7fd29b None None] MFA Rules not processed for user `e0afcee881974443a053c0a5eaad3314`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:01.176059 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f80e5434-76a3-4f92-a6f6-09e29b7fd29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:01.240697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1362/2739] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:01.245149 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:01.309116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a780d-d952-4de1-ab36-2af3b4f82465 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:01.313778 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:01.314393 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.314676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.314957 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.316875 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:01.322152 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:01.326789 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:01.449470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1378/2740] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:01 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:01.453432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5434-76a3-4f92-a6f6-09e29b7fd29b None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:01.464536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e5434-76a3-4f92-a6f6-09e29b7fd29b None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:01.521379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:01.522057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.522258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.522530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.523204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:01.525169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:01.603389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1363/2741] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:01 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:01.607559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:01.619027 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff010-bba7-45ee-baf9-bba629309a18 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:01.623785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:01.624429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.624717 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.624995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.625704 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:01.627330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:01.638117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1379/2742] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:01.642787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:01.652953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017759c-36e2-47e8-a818-5a7b0a0ea404 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:01.660984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:01.662089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.662380 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.663265 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] PATH_INFO: `/v3/projects/9f9efac2aaec48c8a6054ed5941ed273/users/935708fabd58434bbd019bbd5c94fd77/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.683066 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9efac2aaec48c8a6054ed5941ed273, user_id=935708fabd58434bbd019bbd5c94fd77, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:01.685731 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:01.739974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1364/2743] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:01 2022] PUT /identity/v3/projects/9f9efac2aaec48c8a6054ed5941ed273/users/935708fabd58434bbd019bbd5c94fd77/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:01.743696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:01.754691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d62f1-88b4-4cdd-b951-1df290f8ac17 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:01.811185 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112e094-443d-4df1-aa71-9c44c520d93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:01.811801 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112e094-443d-4df1-aa71-9c44c520d93c admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.812117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112e094-443d-4df1-aa71-9c44c520d93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.812403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112e094-443d-4df1-aa71-9c44c520d93c admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.813147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112e094-443d-4df1-aa71-9c44c520d93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:01.815386 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112e094-443d-4df1-aa71-9c44c520d93c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:01.825024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1380/2744] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:01.829316 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:01.840105 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bc00e-5653-49ab-9f50-e5ad7c5bee91 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:01.845580 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:01.846263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.846543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.846819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] PATH_INFO: `/v3/projects/9f9efac2aaec48c8a6054ed5941ed273/users/935708fabd58434bbd019bbd5c94fd77/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.849003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9efac2aaec48c8a6054ed5941ed273, user_id=935708fabd58434bbd019bbd5c94fd77, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:01.854842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:01.857402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:01.863135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1365/2745] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:01 2022] PUT /identity/v3/projects/9f9efac2aaec48c8a6054ed5941ed273/users/935708fabd58434bbd019bbd5c94fd77/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:01.873005 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd8ec6-c867-4d75-a706-1607ab2ba583 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:01.873289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd8ec6-c867-4d75-a706-1607ab2ba583 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:01.873565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd8ec6-c867-4d75-a706-1607ab2ba583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:01.901542 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b0fd8ec6-c867-4d75-a706-1607ab2ba583 None None] MFA Rules not processed for user `935708fabd58434bbd019bbd5c94fd77`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:01.902749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b0fd8ec6-c867-4d75-a706-1607ab2ba583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:01.970283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1381/2746] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:01.984017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:02.040190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3178d7b-c114-42cf-8594-358d774b4120 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:02.043999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:02.044647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:02.044968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:02.045302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:02.054893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:02.056323 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:02.066287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1366/2747] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:02.787412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8349acf-5b65-4963-b8da-889e57056e09 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:02.787930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8349acf-5b65-4963-b8da-889e57056e09 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:02.787930 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8349acf-5b65-4963-b8da-889e57056e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:02.807654 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-f8349acf-5b65-4963-b8da-889e57056e09 None None] MFA Rules not processed for user `bffda1c5c6644b98a4b46c45d5e80a73`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:02.808892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f8349acf-5b65-4963-b8da-889e57056e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:02.861777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:02.876357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1382/2748] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:03.100959 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:03.167343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979c1ed-628a-4adc-8bb0-b8eb6a201c0c service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:03.171996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:03.172671 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:03.172955 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:03.173231 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] PATH_INFO: `/v3/users/e0afcee881974443a053c0a5eaad3314` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:03.174546 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0afcee881974443a053c0a5eaad3314)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:03.177169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:03.326725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0afcee881974443a053c0a5eaad3314'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:03.327389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0afcee881974443a053c0a5eaad3314'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:03.418885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0afcee881974443a053c0a5eaad3314'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:03.431625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0afcee881974443a053c0a5eaad3314'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:03.438479 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0afcee881974443a053c0a5eaad3314'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:03.444881 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1367/2749] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:03 2022] DELETE /identity/v3/users/e0afcee881974443a053c0a5eaad3314 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:03.449449 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:03.449733 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:03.450008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:03.472995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:03.472995 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:03.542641 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1383/2750] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:03.557639 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:03.559333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:03.582985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:03.620940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a17f35-0e92-4451-9dcf-d8f4dfef8a0c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:03.625090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:03.625727 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:03.628141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:03.628141 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:03.636689 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:03.638111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:03.648507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1368/2751] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:04.027432 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:04.029222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:04.049692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed26a6b-b89b-4f6a-8a50-b3fe2f7261be None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:04.056159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:04.056159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:04.056159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:04.056159 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] PATH_INFO: `/v3/projects/5bda1b7e36df4f82b16176476faa60b7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:04.057130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bda1b7e36df4f82b16176476faa60b7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:04.059029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:04.163741 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bda1b7e36df4f82b16176476faa60b7', 'request_id': 'req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.170918 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bda1b7e36df4f82b16176476faa60b7', 'request_id': 'req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.177401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bda1b7e36df4f82b16176476faa60b7', 'request_id': 'req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.299621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] The token cache is being invalidate because project 5bda1b7e36df4f82b16176476faa60b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:04.299913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.300580 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.302145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1384/2752] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:04 2022] DELETE /identity/v3/projects/5bda1b7e36df4f82b16176476faa60b7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:04.306222 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:04.315997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506add65-829e-4929-a828-e7a680412d67 service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:04.371821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:04.372491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:04.372775 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:04.373049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] PATH_INFO: `/v3/users/935708fabd58434bbd019bbd5c94fd77` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:04.374206 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=935708fabd58434bbd019bbd5c94fd77)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:04.376116 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:04.616212 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935708fabd58434bbd019bbd5c94fd77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.617003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935708fabd58434bbd019bbd5c94fd77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.710799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935708fabd58434bbd019bbd5c94fd77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.723776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935708fabd58434bbd019bbd5c94fd77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.731414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d69b5ac4-0a0a-4642-b1b5-48592bfabc87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935708fabd58434bbd019bbd5c94fd77'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:04.737692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1369/2753] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:04 2022] DELETE /identity/v3/users/935708fabd58434bbd019bbd5c94fd77 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:05.118779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:05.120283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:05.133853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16bee5-186c-4c32-8e1a-220c1c7d2b70 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:05.192607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:05.193256 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.193540 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.193824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] PATH_INFO: `/v3/projects/9f9efac2aaec48c8a6054ed5941ed273` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.194998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9efac2aaec48c8a6054ed5941ed273)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:05.196925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:05.283876 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:05.300015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9efac2aaec48c8a6054ed5941ed273', 'request_id': 'req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:05.306002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9efac2aaec48c8a6054ed5941ed273', 'request_id': 'req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:05.311443 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9efac2aaec48c8a6054ed5941ed273', 'request_id': 'req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:05.390355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] The token cache is being invalidate because project 9f9efac2aaec48c8a6054ed5941ed273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:05.390579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:05.391203 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-bb1e0ee1-f6f8-4a8a-8e7a-896ee7998e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:05.392753 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1385/2754] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:05 2022] DELETE /identity/v3/projects/9f9efac2aaec48c8a6054ed5941ed273 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:05.397712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3484b9-5d47-4fb0-9829-1fb8938a0bd1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.398196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3484b9-5d47-4fb0-9829-1fb8938a0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.402856 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3484b9-5d47-4fb0-9829-1fb8938a0bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.417686 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-0f3484b9-5d47-4fb0-9829-1fb8938a0bd1 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:05.418894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-0f3484b9-5d47-4fb0-9829-1fb8938a0bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:05.458003 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:05.478914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1370/2755] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:05.488842 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096371b-9099-463b-92fa-7038d2346a99 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.489127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096371b-9099-463b-92fa-7038d2346a99 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.489428 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096371b-9099-463b-92fa-7038d2346a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.509819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-a096371b-9099-463b-92fa-7038d2346a99 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:05.511150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a096371b-9099-463b-92fa-7038d2346a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:05.519140 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1386/2756] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:05.523438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3484b9-5d47-4fb0-9829-1fb8938a0bd1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:05.534025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3484b9-5d47-4fb0-9829-1fb8938a0bd1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:05.539543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:05.540207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.540515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.540791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.541495 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:53:05.541852 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:05.543473 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:05.555781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1371/2757] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:53:05.560036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096371b-9099-463b-92fa-7038d2346a99 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:05.569568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096371b-9099-463b-92fa-7038d2346a99 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:05.574422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:05.575029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.575309 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.579722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.579722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:05.579722 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:05.689635 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1387/2758] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:05 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:53:05.694006 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:05.704055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18421d4d-f79c-4760-8500-fb6dbab953f5 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:05.767677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:05.768592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.769073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.769684 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.770565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:05.772914 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:05.854706 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1372/2759] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:05 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:05.858629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:05.870680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b40b5-0b29-4de5-bcaf-4e977c8585d8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:05.875507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:05.876124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.876470 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.876744 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.877445 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:05.879041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:05.890276 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1388/2760] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:05.894934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:05.905397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba6508b-f29b-4762-acee-b9e66bd42581 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:05.910676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29a0c7-99f5-4c5c-b8bf-43f553c3fe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:05.911506 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29a0c7-99f5-4c5c-b8bf-43f553c3fe0b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.911890 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29a0c7-99f5-4c5c-b8bf-43f553c3fe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.912172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29a0c7-99f5-4c5c-b8bf-43f553c3fe0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.912972 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29a0c7-99f5-4c5c-b8bf-43f553c3fe0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:05.914598 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29a0c7-99f5-4c5c-b8bf-43f553c3fe0b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:05.925950 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1373/2761] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:05.930148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:05.941195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd5e9a-f280-4146-b103-9be8d59e0f87 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:05.945976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:05.946639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:05.946917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:05.947194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] PATH_INFO: `/v3/projects/8bd3d855ad0b4fd598e6024b19a46870/users/7ddbd9d8bb7045a68c260fe606a040ef/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:05.962925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd3d855ad0b4fd598e6024b19a46870, user_id=7ddbd9d8bb7045a68c260fe606a040ef, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:05.965608 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:06.037427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1389/2762] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:05 2022] PUT /identity/v3/projects/8bd3d855ad0b4fd598e6024b19a46870/users/7ddbd9d8bb7045a68c260fe606a040ef/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:06.047255 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ee09e-f432-4463-a611-108de3de7e76 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.047543 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ee09e-f432-4463-a611-108de3de7e76 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.047820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ee09e-f432-4463-a611-108de3de7e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.076010 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-962ee09e-f432-4463-a611-108de3de7e76 None None] MFA Rules not processed for user `7ddbd9d8bb7045a68c260fe606a040ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:06.080161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-962ee09e-f432-4463-a611-108de3de7e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:06.143553 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1374/2763] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:06.148330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:06.218429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387749ce-1e69-4dbe-988e-7a1bc5ac2db8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:06.223301 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:06.223984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.224265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.224567 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.225239 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:06.227145 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:06.358572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1390/2764] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:06 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:06.365556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ee09e-f432-4463-a611-108de3de7e76 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:06.373673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ee09e-f432-4463-a611-108de3de7e76 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:06.407761 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:06.431104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:06.431663 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.431945 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.432218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.432998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:06.435199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:06.556791 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1375/2765] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:06 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:06.560725 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:06.571669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1b25-5e38-40e8-9768-f406a8b2034a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:06.576375 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:06.577777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.577777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.577777 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.578235 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:06.579858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:06.590655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1391/2766] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:53:06.594771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:06.606608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958dd4b-3060-4c5d-9fd5-a110b07fd00d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:06.611224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca519c9-2f2a-4d37-8994-c4e29c51c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:06.612008 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca519c9-2f2a-4d37-8994-c4e29c51c471 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.612319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca519c9-2f2a-4d37-8994-c4e29c51c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.612640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca519c9-2f2a-4d37-8994-c4e29c51c471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.613350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca519c9-2f2a-4d37-8994-c4e29c51c471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:06.614948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca519c9-2f2a-4d37-8994-c4e29c51c471 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:06.625425 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1376/2767] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:06.630365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:06.640535 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180e3ee-50de-4ad8-b346-8a9f51179fe4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:06.645391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:06.646230 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.646390 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.646973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] PATH_INFO: `/v3/projects/1b7635873bf0416f96e903d6ed8cb50a/users/be62ece81f3946f783b1cbea0a939bb4/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.662062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7635873bf0416f96e903d6ed8cb50a, user_id=be62ece81f3946f783b1cbea0a939bb4, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:06.664743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:06.763731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:06.768693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1392/2768] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:06 2022] PUT /identity/v3/projects/1b7635873bf0416f96e903d6ed8cb50a/users/be62ece81f3946f783b1cbea0a939bb4/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:06.778737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726484ff-fcfe-4de3-a9c1-7e5a5b29626a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.779069 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726484ff-fcfe-4de3-a9c1-7e5a5b29626a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.779353 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726484ff-fcfe-4de3-a9c1-7e5a5b29626a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.817839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-726484ff-fcfe-4de3-a9c1-7e5a5b29626a None None] MFA Rules not processed for user `be62ece81f3946f783b1cbea0a939bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:06.819135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-726484ff-fcfe-4de3-a9c1-7e5a5b29626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:06.887550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1377/2769] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:06.892130 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:06.955559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e49a5d-f903-465b-baee-0ea0d7ac17f9 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:06.960127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:06.961348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:06.961348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:06.961348 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:06.961988 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:06.963865 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:07.086871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1393/2770] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:06 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:07.087896 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726484ff-fcfe-4de3-a9c1-7e5a5b29626a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:07.099621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726484ff-fcfe-4de3-a9c1-7e5a5b29626a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:07.158887 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:07.159515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:07.159819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:07.160074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:07.160787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:07.162692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:07.243492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1378/2771] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:07 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:07.247613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:07.257816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f188d-5deb-46ae-b159-c5ecc3e8eecb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:07.262383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:07.262986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:07.263303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:07.263541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:07.264226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:07.265833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:07.276117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1394/2772] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:07.280190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:07.290672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f21fe-ed01-4f72-9c95-ceb2ea83a197 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:07.295389 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:07.296057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:07.296337 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:07.296645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] PATH_INFO: `/v3/projects/317cde71c53c4b3e8899c55bba187488/users/c76414adfa3a4d5489d469bb0dbae832/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:07.312632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=317cde71c53c4b3e8899c55bba187488, user_id=c76414adfa3a4d5489d469bb0dbae832, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:07.315632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:07.403497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1379/2773] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:07 2022] PUT /identity/v3/projects/317cde71c53c4b3e8899c55bba187488/users/c76414adfa3a4d5489d469bb0dbae832/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:07.407637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:07.418817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb6667-9e18-4910-b4a2-0104a644f0d9 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:07.478253 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667c5f7-26f5-4933-98ae-2aa7fa9ed3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:07.479072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667c5f7-26f5-4933-98ae-2aa7fa9ed3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:07.479454 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667c5f7-26f5-4933-98ae-2aa7fa9ed3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:07.479839 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667c5f7-26f5-4933-98ae-2aa7fa9ed3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:07.480660 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667c5f7-26f5-4933-98ae-2aa7fa9ed3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:07.482343 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667c5f7-26f5-4933-98ae-2aa7fa9ed3ef admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:07.494392 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1395/2774] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:07.497263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:07.507800 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6c125-445d-4fe7-9474-f18c1b38edf3 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:07.512550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:07.513275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:07.513563 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:07.513839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] PATH_INFO: `/v3/projects/317cde71c53c4b3e8899c55bba187488/users/c76414adfa3a4d5489d469bb0dbae832/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:07.515828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=317cde71c53c4b3e8899c55bba187488, user_id=c76414adfa3a4d5489d469bb0dbae832, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:07.519001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:07.524511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:07.529773 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1380/2775] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:07 2022] PUT /identity/v3/projects/317cde71c53c4b3e8899c55bba187488/users/c76414adfa3a4d5489d469bb0dbae832/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:07.541459 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:07.541746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:07.542024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:07.569001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] MFA Rules not processed for user `c76414adfa3a4d5489d469bb0dbae832`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:07.570199 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:07.639886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:07.650730 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1396/2776] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:07.663967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:07.665728 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:07.725442 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d04f-212f-4742-a1cc-b0e12b48cd16 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:07.729368 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:07.729996 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:07.730281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:07.730557 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:07.740544 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:07.741993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:07.752272 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1381/2777] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:53:08.679098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:08.680538 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:08.692678 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ac5ed-3b38-4e54-9e1a-555f2d10eb63 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:08.696242 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:08.696896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:08.697181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:08.697453 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:08.753344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:08.754699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:08.763484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1397/2778] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:13.612099 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:13.621021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e555b45-a38d-4e4a-85ce-5fca7048c505 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:13.624855 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7f0cd-3ec3-443a-8118-0384cf3bdad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:13.625460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f0cd-3ec3-443a-8118-0384cf3bdad1 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:13.625743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f0cd-3ec3-443a-8118-0384cf3bdad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:13.626668 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f0cd-3ec3-443a-8118-0384cf3bdad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:13.682205 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7f0cd-3ec3-443a-8118-0384cf3bdad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:13.683620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7f0cd-3ec3-443a-8118-0384cf3bdad1 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:13.692907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1382/2779] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:17.357494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:17.423129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb6532-03e6-4a3e-8c19-fac2c5c0d119 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:17.427554 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:17.428292 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:17.428598 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:17.428869 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] PATH_INFO: `/v3/users/7ddbd9d8bb7045a68c260fe606a040ef` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:17.430002 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ddbd9d8bb7045a68c260fe606a040ef)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:17.431871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:17.592812 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ddbd9d8bb7045a68c260fe606a040ef'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:17.593523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ddbd9d8bb7045a68c260fe606a040ef'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:17.653278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ddbd9d8bb7045a68c260fe606a040ef'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:17.666726 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ddbd9d8bb7045a68c260fe606a040ef'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:17.667404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ddbd9d8bb7045a68c260fe606a040ef'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:17.680706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1398/2780] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:17 2022] DELETE /identity/v3/users/7ddbd9d8bb7045a68c260fe606a040ef => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:17.685226 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:17.685735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:17.685865 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:17.704306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:17.705540 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:17.764123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1383/2781] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:17.773858 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:17.775251 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:17.790195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:17.833686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c524d-abfe-4775-8b6f-24e66277dcd5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:17.837314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:17.837926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:17.838208 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:17.838485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:17.849446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:17.851146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:17.861926 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1399/2782] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:18.242873 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:18.244304 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:18.256647 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19ed71-3f32-4b5d-a70d-915ee747673c None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:18.261444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:18.262081 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:18.262364 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:18.262640 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] PATH_INFO: `/v3/projects/8bd3d855ad0b4fd598e6024b19a46870` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:18.263886 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd3d855ad0b4fd598e6024b19a46870)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:18.268333 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:18.376494 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:18.385783 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd3d855ad0b4fd598e6024b19a46870', 'request_id': 'req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.392453 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd3d855ad0b4fd598e6024b19a46870', 'request_id': 'req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.397917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd3d855ad0b4fd598e6024b19a46870', 'request_id': 'req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.487701 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] The token cache is being invalidate because project 8bd3d855ad0b4fd598e6024b19a46870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:18.491239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.491239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.491239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1384/2783] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:18 2022] DELETE /identity/v3/projects/8bd3d855ad0b4fd598e6024b19a46870 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:18.494146 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:18.504477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738e7fc-f805-4125-82be-0de257779807 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:18.561509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:18.562169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:18.562431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:18.562708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] PATH_INFO: `/v3/users/be62ece81f3946f783b1cbea0a939bb4` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:18.564582 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] RBAC: Authorizing `identity:delete_user(user_id=be62ece81f3946f783b1cbea0a939bb4)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:18.566586 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:18.839007 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be62ece81f3946f783b1cbea0a939bb4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.839676 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be62ece81f3946f783b1cbea0a939bb4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.921845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be62ece81f3946f783b1cbea0a939bb4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.936036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be62ece81f3946f783b1cbea0a939bb4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.941424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be62ece81f3946f783b1cbea0a939bb4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:18.949827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1400/2784] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:18 2022] DELETE /identity/v3/users/be62ece81f3946f783b1cbea0a939bb4 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:19.449156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:19.450827 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:19.464982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8b2bb-821f-4838-80a1-d9cc31a4fc9f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:19.521584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:19.522227 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:19.522508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:19.522780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] PATH_INFO: `/v3/projects/1b7635873bf0416f96e903d6ed8cb50a` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:19.523940 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b7635873bf0416f96e903d6ed8cb50a)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:19.525869 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:19.635616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b7635873bf0416f96e903d6ed8cb50a', 'request_id': 'req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:19.642243 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b7635873bf0416f96e903d6ed8cb50a', 'request_id': 'req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:19.647680 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b7635873bf0416f96e903d6ed8cb50a', 'request_id': 'req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:19.738551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] The token cache is being invalidate because project 1b7635873bf0416f96e903d6ed8cb50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:19.738843 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:19.739480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:19.741060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1385/2785] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:19 2022] DELETE /identity/v3/projects/1b7635873bf0416f96e903d6ed8cb50a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:19.745105 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:19.746525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:19.759579 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20faf6e-5a74-4a37-a758-79269c2afcea admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:19.791431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:19.814863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:19.815496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:19.815779 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:19.816056 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] PATH_INFO: `/v3/users/c76414adfa3a4d5489d469bb0dbae832` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:19.817272 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76414adfa3a4d5489d469bb0dbae832)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:19.819529 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:19.969176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76414adfa3a4d5489d469bb0dbae832'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:19.969843 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76414adfa3a4d5489d469bb0dbae832'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.030425 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76414adfa3a4d5489d469bb0dbae832'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.043528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76414adfa3a4d5489d469bb0dbae832'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.044247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-610b6c7c-711a-4840-ae77-cb9f5c468d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76414adfa3a4d5489d469bb0dbae832'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.056999 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1401/2786] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:19 2022] DELETE /identity/v3/users/c76414adfa3a4d5489d469bb0dbae832 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:20.505821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:20.507282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:20.527625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241059f4-b5a2-4fa9-8b07-1fdd3955e34f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:20.575326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:20.584110 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:20.584781 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:20.585061 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:20.585356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] PATH_INFO: `/v3/projects/317cde71c53c4b3e8899c55bba187488` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:20.586514 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] RBAC: Authorizing `identity:delete_project(project_id=317cde71c53c4b3e8899c55bba187488)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:20.588439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:20.706974 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317cde71c53c4b3e8899c55bba187488', 'request_id': 'req-ece2f037-efd1-49e5-9a1a-d7d6096d523b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.713426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317cde71c53c4b3e8899c55bba187488', 'request_id': 'req-ece2f037-efd1-49e5-9a1a-d7d6096d523b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.720160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317cde71c53c4b3e8899c55bba187488', 'request_id': 'req-ece2f037-efd1-49e5-9a1a-d7d6096d523b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.772648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] The token cache is being invalidate because project 317cde71c53c4b3e8899c55bba187488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:20.772946 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.773571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-ece2f037-efd1-49e5-9a1a-d7d6096d523b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:20.775123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1386/2787] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:20 2022] DELETE /identity/v3/projects/317cde71c53c4b3e8899c55bba187488 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:20.781008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eecb61-9cde-4799-8c8e-92dafe2d97fb None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:20.781494 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eecb61-9cde-4799-8c8e-92dafe2d97fb None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:20.781610 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eecb61-9cde-4799-8c8e-92dafe2d97fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:20.802139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-04eecb61-9cde-4799-8c8e-92dafe2d97fb None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:20.803273 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-04eecb61-9cde-4799-8c8e-92dafe2d97fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:20.864001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1402/2788] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:20.872614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b638082-b6de-4bdc-b8e5-534b0d0a8556 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:20.872904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b638082-b6de-4bdc-b8e5-534b0d0a8556 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:20.873184 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b638082-b6de-4bdc-b8e5-534b0d0a8556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:20.893837 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6b638082-b6de-4bdc-b8e5-534b0d0a8556 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:20.895083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6b638082-b6de-4bdc-b8e5-534b0d0a8556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:20.903297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1387/2789] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:53:20.908110 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eecb61-9cde-4799-8c8e-92dafe2d97fb None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:20.917759 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eecb61-9cde-4799-8c8e-92dafe2d97fb None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:20.922335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:20.922941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:20.923224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:20.923497 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:20.924186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:53:20.924523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:20.926106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:20.937368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1403/2790] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:53:20.942363 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b638082-b6de-4bdc-b8e5-534b0d0a8556 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:20.950754 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b638082-b6de-4bdc-b8e5-534b0d0a8556 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:20.955838 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:20.956462 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:20.956749 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:20.957018 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:20.957685 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:20.959545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:21.064734 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1388/2791] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:20 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:21.066441 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:21.076264 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ceb5b-92e2-4405-8c37-bdf7202974b8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:21.120989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:21.136761 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:21.137414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:21.137698 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:21.137975 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:21.138958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:21.141772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:21.227373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1404/2792] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:21 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:21.229642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:21.240694 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db287bc-ce8a-4b9f-9b81-5f0a9c7fe8cf admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:21.245489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:21.246098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:21.246381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:21.246658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:21.248236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:21.250241 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:21.261158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1389/2793] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:21.265293 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:21.275847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f53ca-6c5b-48ef-877a-b894508442db admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:21.280448 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b472ad6-e820-4419-9a85-59c51c86331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:21.281381 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472ad6-e820-4419-9a85-59c51c86331a admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:21.283963 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472ad6-e820-4419-9a85-59c51c86331a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:21.284247 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472ad6-e820-4419-9a85-59c51c86331a admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:21.284992 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b472ad6-e820-4419-9a85-59c51c86331a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:21.286574 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b472ad6-e820-4419-9a85-59c51c86331a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:21.297129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1405/2794] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:21.301338 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:21.311480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25c1e1-390e-43c6-9cf6-3fd79e1a2c83 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:21.319459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:21.319459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:21.319459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:21.319459 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] PATH_INFO: `/v3/projects/6221157750b74125a5f88391a54244af/users/3690927e0a0b4ae3afd965e6d24a0fd3/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:21.335135 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6221157750b74125a5f88391a54244af, user_id=3690927e0a0b4ae3afd965e6d24a0fd3, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:21.338194 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:21.432710 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1390/2795] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:21 2022] PUT /identity/v3/projects/6221157750b74125a5f88391a54244af/users/3690927e0a0b4ae3afd965e6d24a0fd3/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:21.437743 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876690f-537b-43c9-a356-c0ac3f6fa34c None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:21.438502 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876690f-537b-43c9-a356-c0ac3f6fa34c None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:21.438790 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876690f-537b-43c9-a356-c0ac3f6fa34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:21.458158 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-5876690f-537b-43c9-a356-c0ac3f6fa34c None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:21.459373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5876690f-537b-43c9-a356-c0ac3f6fa34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:21.519793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1406/2796] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:21.529287 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:21.531224 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:21.597633 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d929a4-da11-45af-afe0-ed22a89e546f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:21.601356 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cff54-41de-463b-971c-7ae8734c3634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:21.601964 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cff54-41de-463b-971c-7ae8734c3634 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:21.602244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cff54-41de-463b-971c-7ae8734c3634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:21.602517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cff54-41de-463b-971c-7ae8734c3634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:21.612264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cff54-41de-463b-971c-7ae8734c3634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:21.613714 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cff54-41de-463b-971c-7ae8734c3634 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:21.623012 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1391/2797] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:22.476015 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c2f1-a2c7-475b-8b67-eb9aa114d849 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:22.476564 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c2f1-a2c7-475b-8b67-eb9aa114d849 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:22.476680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28c2f1-a2c7-475b-8b67-eb9aa114d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:22.495385 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-be28c2f1-a2c7-475b-8b67-eb9aa114d849 None None] MFA Rules not processed for user `bffda1c5c6644b98a4b46c45d5e80a73`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:22.496679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-be28c2f1-a2c7-475b-8b67-eb9aa114d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:22.551291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1407/2798] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:23.206017 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6b36e-9465-4f61-b6f6-27eb5d1379d1 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:23.206545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6b36e-9465-4f61-b6f6-27eb5d1379d1 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:23.206545 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6b36e-9465-4f61-b6f6-27eb5d1379d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:23.231158 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:23.234488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-1ad6b36e-9465-4f61-b6f6-27eb5d1379d1 None None] MFA Rules not processed for user `3690927e0a0b4ae3afd965e6d24a0fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:23.236217 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1ad6b36e-9465-4f61-b6f6-27eb5d1379d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:23.305325 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1392/2799] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:23.309675 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28c2f1-a2c7-475b-8b67-eb9aa114d849 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:23.323916 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28c2f1-a2c7-475b-8b67-eb9aa114d849 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:23.328486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:23.329125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:23.329404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:23.329677 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:23.330337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:23.332307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:23.419984 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1408/2800] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:23 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:23.424693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6b36e-9465-4f61-b6f6-27eb5d1379d1 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:23.437030 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6b36e-9465-4f61-b6f6-27eb5d1379d1 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:23.498457 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:23.499037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:23.499319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:23.499842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:23.501202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:23.503242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:23.591275 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1393/2801] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:23 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:53:23.595661 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:23.606525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4af75-bdce-4c22-9ef2-6ba99d107061 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:23.612133 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:23.612853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:23.613137 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:23.613417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:23.614108 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:23.615692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:23.625625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1409/2802] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:23.630056 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:23.640636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e82e0-652b-46ca-94ad-499f45cf858a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:23.645252 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:23.645923 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:23.646198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:23.646471 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] PATH_INFO: `/v3/projects/aa88c97a480648c59397fa2abe6055f7/users/7cf49ad7bddd4bd5abf78fd82451d620/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:23.662830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa88c97a480648c59397fa2abe6055f7, user_id=7cf49ad7bddd4bd5abf78fd82451d620, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:23.665497 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:23.744556 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1394/2803] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:23 2022] PUT /identity/v3/projects/aa88c97a480648c59397fa2abe6055f7/users/7cf49ad7bddd4bd5abf78fd82451d620/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:23.748881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:23.759885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28265edd-96ce-40c7-9e63-e58f180e5638 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:23.790112 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:23.821559 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaec4cf-48c1-412e-9d1d-d769733b174d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:23.822252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaec4cf-48c1-412e-9d1d-d769733b174d admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:23.822530 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaec4cf-48c1-412e-9d1d-d769733b174d admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:23.822800 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaec4cf-48c1-412e-9d1d-d769733b174d admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:23.823484 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaec4cf-48c1-412e-9d1d-d769733b174d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:23.825129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaec4cf-48c1-412e-9d1d-d769733b174d admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:23.835211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1410/2804] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:23.839620 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:23.851237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f62e2-35be-472d-987c-49f9eee244ce admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:23.856175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:23.856863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:23.857139 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:23.857410 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] PATH_INFO: `/v3/projects/aa88c97a480648c59397fa2abe6055f7/users/7cf49ad7bddd4bd5abf78fd82451d620/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:23.859690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa88c97a480648c59397fa2abe6055f7, user_id=7cf49ad7bddd4bd5abf78fd82451d620, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:23.862365 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:23.872372 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1395/2805] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:23 2022] PUT /identity/v3/projects/aa88c97a480648c59397fa2abe6055f7/users/7cf49ad7bddd4bd5abf78fd82451d620/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:25.174618 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:25.175194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:25.175194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:25.204642 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] MFA Rules not processed for user `7cf49ad7bddd4bd5abf78fd82451d620`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:25.205878 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:25.276278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1411/2806] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:25.288440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:25.290739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:25.348068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:25.357379 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f191c-b6a9-49d4-8cc9-857428a861ba admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:25.361371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:25.362014 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:25.362851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:25.362851 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:25.419221 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:25.420582 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:25.429452 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1396/2807] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:27.226885 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:27.228294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:27.240627 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8521b3-e6d7-4503-aa0d-892428363fcc None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:27.244297 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:27.244965 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:27.245248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:27.245523 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:27.255483 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:27.256951 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:27.267771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1412/2808] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:35.516214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=28 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:36.559035 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:36.615083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefec85-7808-4873-adf7-77827a200cc0 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:36.622340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:36.622340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:36.622340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:36.622340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:36.622729 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:36.702004 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:36.703552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:36.723809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1397/2809] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:37.262278 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=56 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:37.896851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:37.906699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30f8b2-ba5c-4b3c-9307-6767c428b4d8 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:37.910317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:37.910922 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:37.911204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:37.911480 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:37.913261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:37.927527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:37.928987 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:37.940303 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1413/2810] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:43.093023 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:43.157313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90e8ee-4245-46fc-9600-f8e3ff5d0cff service neutron] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:43.161771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:43.162402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:43.162681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:43.162951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] PATH_INFO: `/v3/users/3690927e0a0b4ae3afd965e6d24a0fd3` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:43.164126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3690927e0a0b4ae3afd965e6d24a0fd3)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:43.166064 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:43.259245 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:43.385986 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3690927e0a0b4ae3afd965e6d24a0fd3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:43.386648 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3690927e0a0b4ae3afd965e6d24a0fd3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:43.451571 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3690927e0a0b4ae3afd965e6d24a0fd3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:43.464326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3690927e0a0b4ae3afd965e6d24a0fd3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:43.472088 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3690927e0a0b4ae3afd965e6d24a0fd3'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:43.478111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1398/2811] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:43 2022] DELETE /identity/v3/users/3690927e0a0b4ae3afd965e6d24a0fd3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:43.785039 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:43.796616 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077b736-a492-4dc6-b95b-52971e6ad1ca service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:43.859147 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:43.859838 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:43.860124 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:43.860422 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] PATH_INFO: `/v3/projects/6221157750b74125a5f88391a54244af` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:43.861587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6221157750b74125a5f88391a54244af)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:43.863479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:43.973113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6221157750b74125a5f88391a54244af', 'request_id': 'req-1a6c8660-eee6-4721-b616-125fa8a48327'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:43.979463 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6221157750b74125a5f88391a54244af', 'request_id': 'req-1a6c8660-eee6-4721-b616-125fa8a48327'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:43.985404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6221157750b74125a5f88391a54244af', 'request_id': 'req-1a6c8660-eee6-4721-b616-125fa8a48327'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.063506 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] The token cache is being invalidate because project 6221157750b74125a5f88391a54244af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:44.063731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.064331 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.065892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1414/2812] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:43 2022] DELETE /identity/v3/projects/6221157750b74125a5f88391a54244af => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:44.070123 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:44.073894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:44.084548 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccbbdc-41b5-4949-a8f8-1b6c6a96743a admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:44.145143 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:44.145839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:44.146054 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:44.146621 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] PATH_INFO: `/v3/users/7cf49ad7bddd4bd5abf78fd82451d620` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:44.147804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf49ad7bddd4bd5abf78fd82451d620)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:44.150022 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:44.334414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf49ad7bddd4bd5abf78fd82451d620'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.335001 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf49ad7bddd4bd5abf78fd82451d620'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.399806 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf49ad7bddd4bd5abf78fd82451d620'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.412757 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf49ad7bddd4bd5abf78fd82451d620'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.419477 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-d1522c35-b7f0-4919-9dcb-856cbe7b1f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf49ad7bddd4bd5abf78fd82451d620'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.426057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1399/2813] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:44 2022] DELETE /identity/v3/users/7cf49ad7bddd4bd5abf78fd82451d620 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:44.694050 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:44.695505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:44.708155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c8660-eee6-4721-b616-125fa8a48327 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:44.763989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:44.764697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:44.764982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:44.765261 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] PATH_INFO: `/v3/projects/aa88c97a480648c59397fa2abe6055f7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:44.767029 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa88c97a480648c59397fa2abe6055f7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:44.769013 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:44.790404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:44.869612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa88c97a480648c59397fa2abe6055f7', 'request_id': 'req-5117b0cf-8721-4579-ba0b-21261b8e9ce4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.876706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa88c97a480648c59397fa2abe6055f7', 'request_id': 'req-5117b0cf-8721-4579-ba0b-21261b8e9ce4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.882308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa88c97a480648c59397fa2abe6055f7', 'request_id': 'req-5117b0cf-8721-4579-ba0b-21261b8e9ce4'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.965664 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] The token cache is being invalidate because project aa88c97a480648c59397fa2abe6055f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:44.965888 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.966505 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-5117b0cf-8721-4579-ba0b-21261b8e9ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:44.968035 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1415/2814] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:44 2022] DELETE /identity/v3/projects/aa88c97a480648c59397fa2abe6055f7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:44.972902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5bacd-9f0a-4e09-b05c-52f69d388b78 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:44.973201 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5bacd-9f0a-4e09-b05c-52f69d388b78 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:44.973488 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5bacd-9f0a-4e09-b05c-52f69d388b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:44.993810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-c1f5bacd-9f0a-4e09-b05c-52f69d388b78 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:44.995025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-c1f5bacd-9f0a-4e09-b05c-52f69d388b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:45.014081 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:45.053693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1400/2815] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:45.062487 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061217c-cbb8-4dfe-b5ee-a9e5831dbf48 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.063464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061217c-cbb8-4dfe-b5ee-a9e5831dbf48 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.063773 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061217c-cbb8-4dfe-b5ee-a9e5831dbf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.084623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-c061217c-cbb8-4dfe-b5ee-a9e5831dbf48 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:45.085818 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c061217c-cbb8-4dfe-b5ee-a9e5831dbf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:45.093638 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1416/2816] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:45.097704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5bacd-9f0a-4e09-b05c-52f69d388b78 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.107273 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5bacd-9f0a-4e09-b05c-52f69d388b78 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.112592 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.113190 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.113468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.113740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.114431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:53:45.114735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.116309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.128103 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1401/2817] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:53:45.132224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061217c-cbb8-4dfe-b5ee-a9e5831dbf48 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.141267 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061217c-cbb8-4dfe-b5ee-a9e5831dbf48 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.145828 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.146427 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.146707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.146978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.147637 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.149539 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.224089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1417/2818] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:45 2022] POST /identity/v3/projects => generated 371 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:45.228120 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.238404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570d83b-1b03-49fc-bd11-ca130e77f8e6 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.294891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.295503 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.295787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.296065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.296760 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.298750 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.361060 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1402/2819] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:45 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:45.365286 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.376144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3dfab-d8c4-4741-a949-cd80e3e1bf71 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.380982 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.381771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.381887 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.382150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.382831 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.384451 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.398155 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1418/2820] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:45.405657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.412218 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84da4c-2627-4998-8281-21d39e67f450 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.416993 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f6d0d-662c-4113-8473-f066061b3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.419627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f6d0d-662c-4113-8473-f066061b3959 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.419627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f6d0d-662c-4113-8473-f066061b3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.419627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f6d0d-662c-4113-8473-f066061b3959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.419627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f6d0d-662c-4113-8473-f066061b3959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.420417 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f6d0d-662c-4113-8473-f066061b3959 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.431171 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1403/2821] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:45.435401 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.445877 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b398c57-351d-442a-b1aa-7526afe566cc admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.450458 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.454176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.454176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.454176 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] PATH_INFO: `/v3/projects/84f4946ee8aa4c6492e72b1cae821fc3/users/6182055529914119a573560f68e082d2/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.467042 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f4946ee8aa4c6492e72b1cae821fc3, user_id=6182055529914119a573560f68e082d2, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.469827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.559623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1419/2822] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:45 2022] PUT /identity/v3/projects/84f4946ee8aa4c6492e72b1cae821fc3/users/6182055529914119a573560f68e082d2/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:45.570677 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2098fd8-4ff3-42b8-8389-da19efbdcba3 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.570969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2098fd8-4ff3-42b8-8389-da19efbdcba3 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.571246 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2098fd8-4ff3-42b8-8389-da19efbdcba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.598699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a2098fd8-4ff3-42b8-8389-da19efbdcba3 None None] MFA Rules not processed for user `6182055529914119a573560f68e082d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:45.599920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a2098fd8-4ff3-42b8-8389-da19efbdcba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:45.664755 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1404/2823] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:53:45.679737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.679737 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:45.737626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08c67d-78cd-459a-a6b2-6dea8d0a51b4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.741420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a144adce-acf6-4958-9604-340d2649dec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.742034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144adce-acf6-4958-9604-340d2649dec6 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.742387 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144adce-acf6-4958-9604-340d2649dec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.742658 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a144adce-acf6-4958-9604-340d2649dec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.751431 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144adce-acf6-4958-9604-340d2649dec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.752896 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a144adce-acf6-4958-9604-340d2649dec6 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.761798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1420/2824] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:45.783404 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2098fd8-4ff3-42b8-8389-da19efbdcba3 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:45.806853 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.807332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:45.807446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.807446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.850428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2098fd8-4ff3-42b8-8389-da19efbdcba3 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:45.855708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:45.856541 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:45.856819 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:45.857090 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] PATH_INFO: `/v3/users/6182055529914119a573560f68e082d2` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:45.858831 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6182055529914119a573560f68e082d2)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:45.861268 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:45.867711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:45.868938 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:45.878669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:45.933461 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1421/2825] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:53:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:53:46.051925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:46.053667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:46.055428 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6182055529914119a573560f68e082d2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:46.058207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6182055529914119a573560f68e082d2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:46.143372 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4203f-4a0d-425f-a285-079d19a4aa7e None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:46.148608 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6182055529914119a573560f68e082d2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:46.149078 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f8004-2b08-4b93-9fc3-5b1bd3acd487 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:46.149824 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f8004-2b08-4b93-9fc3-5b1bd3acd487 service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:46.150246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f8004-2b08-4b93-9fc3-5b1bd3acd487 service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:46.150521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f8004-2b08-4b93-9fc3-5b1bd3acd487 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:46.159590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f8004-2b08-4b93-9fc3-5b1bd3acd487 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:46.161211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f8004-2b08-4b93-9fc3-5b1bd3acd487 service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:46.167381 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6182055529914119a573560f68e082d2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:46.169903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1422/2826] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:53:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 20:53:46.175688 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6182055529914119a573560f68e082d2'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:46.182148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1405/2827] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:45 2022] DELETE /identity/v3/users/6182055529914119a573560f68e082d2 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:46.186699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:46.187026 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:46.187270 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:46.210695 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:46.212062 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:46.308752 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1423/2828] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:46.318586 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:46.320733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:46.399319 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d541d-1131-47f3-8c10-b2deb0c36728 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:46.404450 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a94b5-d394-4122-a60c-ff181778e7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:46.405489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a94b5-d394-4122-a60c-ff181778e7ed service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:46.405703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a94b5-d394-4122-a60c-ff181778e7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:46.405979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a94b5-d394-4122-a60c-ff181778e7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:46.416975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a94b5-d394-4122-a60c-ff181778e7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:46.418415 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a94b5-d394-4122-a60c-ff181778e7ed service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:46.431409 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1406/2829] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:53:46.890260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:46.893477 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:46.935326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5585153-43a6-4b59-ba18-dab0bbebca52 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:46.947126 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:46.947924 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:46.948412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:46.948709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] PATH_INFO: `/v3/projects/84f4946ee8aa4c6492e72b1cae821fc3` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:46.950054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f4946ee8aa4c6492e72b1cae821fc3)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:46.952748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:46.972597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:47.071544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f4946ee8aa4c6492e72b1cae821fc3', 'request_id': 'req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:47.080915 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f4946ee8aa4c6492e72b1cae821fc3', 'request_id': 'req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:47.090928 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f4946ee8aa4c6492e72b1cae821fc3', 'request_id': 'req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:47.179845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] The token cache is being invalidate because project 84f4946ee8aa4c6492e72b1cae821fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:47.180162 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:47.180815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-20d2240b-2b99-4c74-ab6f-1d5f9881ea92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:47.186827 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1424/2830] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:46 2022] DELETE /identity/v3/projects/84f4946ee8aa4c6492e72b1cae821fc3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:47.191284 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e21fe-c9ce-42ec-bb44-b5133034a2bb None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.191834 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e21fe-c9ce-42ec-bb44-b5133034a2bb None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.192525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e21fe-c9ce-42ec-bb44-b5133034a2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.235830 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-765e21fe-c9ce-42ec-bb44-b5133034a2bb None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:47.237177 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-765e21fe-c9ce-42ec-bb44-b5133034a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:47.315743 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:47.355796 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1407/2831] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:47.364516 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe76b7-70e4-4c25-9116-f7cb6cda07a0 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.364892 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe76b7-70e4-4c25-9116-f7cb6cda07a0 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.365248 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe76b7-70e4-4c25-9116-f7cb6cda07a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.401509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8bbe76b7-70e4-4c25-9116-f7cb6cda07a0 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:47.401509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8bbe76b7-70e4-4c25-9116-f7cb6cda07a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:47.408006 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1425/2832] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:47.421111 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e21fe-c9ce-42ec-bb44-b5133034a2bb None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:47.462483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e21fe-c9ce-42ec-bb44-b5133034a2bb None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:47.462483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:47.462483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.462483 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.463281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.463281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:53:47.463281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:47.463281 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:47.504799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1408/2833] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:53:47.509749 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe76b7-70e4-4c25-9116-f7cb6cda07a0 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:47.530103 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe76b7-70e4-4c25-9116-f7cb6cda07a0 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:47.536889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:47.537590 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.537894 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.538204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.538925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:47.541008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:47.670528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1426/2834] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:47 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:47.671140 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:47.685505 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa96b32-34e6-473b-a4f1-2fc28e20787c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:47.764332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:47.766708 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.767164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.767569 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.768418 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:47.770801 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:47.866628 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1409/2835] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:47 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:47.871008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:47.883086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6c577-ee08-409f-8136-e528b096bb34 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:47.888260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:47.889019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.889436 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.889822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.890613 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:47.892330 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:47.907207 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:47.916473 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1427/2836] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:47.918796 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70030476-2260-4c95-84b1-0784bc1b62d9 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:47.924657 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24adb2c-61da-49fa-8acd-d0ca08a1b1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:47.925345 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24adb2c-61da-49fa-8acd-d0ca08a1b1be admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.925713 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24adb2c-61da-49fa-8acd-d0ca08a1b1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.926055 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24adb2c-61da-49fa-8acd-d0ca08a1b1be admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.926821 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24adb2c-61da-49fa-8acd-d0ca08a1b1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:47.928499 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24adb2c-61da-49fa-8acd-d0ca08a1b1be admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:47.945753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1410/2837] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:53:47.946860 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:47.959581 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc5a57-63b4-4f74-8dd5-9bc28cd76a96 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:47.966893 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:47.967884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:47.968429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:47.968813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] PATH_INFO: `/v3/projects/907705bd47c64492922c6cdde04c3407/users/c414197f142e4912b05abb8b0c6dc57b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:47.988649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=907705bd47c64492922c6cdde04c3407, user_id=c414197f142e4912b05abb8b0c6dc57b, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:47.991772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:48.063653 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1428/2838] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:47 2022] PUT /identity/v3/projects/907705bd47c64492922c6cdde04c3407/users/c414197f142e4912b05abb8b0c6dc57b/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:53:48.074846 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4479aa-caf0-4966-bfd2-3932f53bfd58 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:48.075516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4479aa-caf0-4966-bfd2-3932f53bfd58 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:48.075795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4479aa-caf0-4966-bfd2-3932f53bfd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:48.116481 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6d4479aa-caf0-4966-bfd2-3932f53bfd58 None None] MFA Rules not processed for user `c414197f142e4912b05abb8b0c6dc57b`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:48.118233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6d4479aa-caf0-4966-bfd2-3932f53bfd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:48.200422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1411/2839] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:53:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:48.213974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:48.215859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:48.282607 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57fd31-0e26-4e4c-97f1-41db0c33d9a8 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:48.286508 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:48.287139 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:48.287420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:48.287709 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:48.297629 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:48.299058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:48.308565 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1429/2840] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:58.197165 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:58.304032 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:07.019065 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:07.019562 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:07.019674 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:07.045725 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:07.047423 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:07.102037 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1412/2841] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 27 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:07.115232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:07.116711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:07.188942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e13ff3-3752-4b51-88f1-5c9291eb03e1 service glance] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:07.193793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:07.194403 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:07.194687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:07.194958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:07.202848 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:07.204478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:07.212337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1430/2842] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:54:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:54:10.133097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:10.134282 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:10.200825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec8721-c242-4454-9b83-5e4486898bfc None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:10.205402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:10.206045 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:10.206327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:10.206609 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] PATH_INFO: `/v3/users/bffda1c5c6644b98a4b46c45d5e80a73` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:10.207797 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] RBAC: Authorizing `identity:delete_user(user_id=bffda1c5c6644b98a4b46c45d5e80a73)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:10.209762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:10.339126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bffda1c5c6644b98a4b46c45d5e80a73'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.340203 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bffda1c5c6644b98a4b46c45d5e80a73'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.399730 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bffda1c5c6644b98a4b46c45d5e80a73'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.413422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bffda1c5c6644b98a4b46c45d5e80a73'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.420508 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2ca1176f-b546-45cc-8f23-2460632e91be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bffda1c5c6644b98a4b46c45d5e80a73'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.427370 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1413/2843] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:10 2022] DELETE /identity/v3/users/bffda1c5c6644b98a4b46c45d5e80a73 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:10.659899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:10.673881 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dbc2c-7c3f-4ece-90be-463f760a935e service swift] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:10.732129 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:10.733048 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:10.733326 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:10.733595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] PATH_INFO: `/v3/projects/ab2db90ccbbf468587d34d323c7bcd8a` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:10.734807 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2db90ccbbf468587d34d323c7bcd8a)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:10.736710 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:10.834280 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2db90ccbbf468587d34d323c7bcd8a', 'request_id': 'req-99c4c540-737b-4a09-b81b-95176257d369'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.840423 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2db90ccbbf468587d34d323c7bcd8a', 'request_id': 'req-99c4c540-737b-4a09-b81b-95176257d369'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.846731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2db90ccbbf468587d34d323c7bcd8a', 'request_id': 'req-99c4c540-737b-4a09-b81b-95176257d369'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.930397 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] The token cache is being invalidate because project ab2db90ccbbf468587d34d323c7bcd8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:10.930712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.931328 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-99c4c540-737b-4a09-b81b-95176257d369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:10.932969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1431/2844] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:10 2022] DELETE /identity/v3/projects/ab2db90ccbbf468587d34d323c7bcd8a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:10.938049 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7b11c-d4b3-4eaf-99f7-fd73298278e8 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:10.938525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7b11c-d4b3-4eaf-99f7-fd73298278e8 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:10.938808 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7b11c-d4b3-4eaf-99f7-fd73298278e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:10.959326 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-cdf7b11c-d4b3-4eaf-99f7-fd73298278e8 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:10.960575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-cdf7b11c-d4b3-4eaf-99f7-fd73298278e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:10.980136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:11.026771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1414/2845] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:11.037407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590b86-aad4-4875-b921-c4b835b32fc3 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.037692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590b86-aad4-4875-b921-c4b835b32fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.037967 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11590b86-aad4-4875-b921-c4b835b32fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.057913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-11590b86-aad4-4875-b921-c4b835b32fc3 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:11.059102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-11590b86-aad4-4875-b921-c4b835b32fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:11.067857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1432/2846] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:11.072157 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7b11c-d4b3-4eaf-99f7-fd73298278e8 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.082484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7b11c-d4b3-4eaf-99f7-fd73298278e8 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.087156 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.087762 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.088039 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.088309 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.089031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:11.089332 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.090984 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.103402 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1415/2847] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:11.107561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11590b86-aad4-4875-b921-c4b835b32fc3 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.117053 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11590b86-aad4-4875-b921-c4b835b32fc3 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.121731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.122332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.122611 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.122884 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.123543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.125955 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.137095 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:11.252728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1433/2848] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:11 2022] POST /identity/v3/projects => generated 397 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:11.257077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.268083 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ecce6-fc2b-4058-b449-3eb4365073ce admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.323634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.324175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.324492 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.324771 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.325437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.327385 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.416875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1416/2849] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:11 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:11.420968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.431588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c7fb6-2ddb-4c47-b29f-b847c2595fdf admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.436232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.436891 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.437169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.444528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.444528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.444528 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.451907 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1434/2850] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:11.455825 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.466512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fe9a2-3a8b-4ebd-9083-0b542a374080 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.471588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3765b6-f0f0-43dd-b029-ae66cde4799f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.472320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3765b6-f0f0-43dd-b029-ae66cde4799f admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.472627 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3765b6-f0f0-43dd-b029-ae66cde4799f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.472941 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3765b6-f0f0-43dd-b029-ae66cde4799f admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.473642 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3765b6-f0f0-43dd-b029-ae66cde4799f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.475440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3765b6-f0f0-43dd-b029-ae66cde4799f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.492344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1417/2851] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:11.505259 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.515696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6c793-0df4-4938-b0d4-aaeddff6b3a0 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.520479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.521169 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.521446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.521719 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] PATH_INFO: `/v3/projects/84c83082a73646afb8d4d33232e14201/users/bf6bc49148e747108a1f1877ddbfae90/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.537014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c83082a73646afb8d4d33232e14201, user_id=bf6bc49148e747108a1f1877ddbfae90, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.539663 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.622420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1435/2852] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:11 2022] PUT /identity/v3/projects/84c83082a73646afb8d4d33232e14201/users/bf6bc49148e747108a1f1877ddbfae90/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:11.633148 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.633434 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.633709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.661631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] MFA Rules not processed for user `bf6bc49148e747108a1f1877ddbfae90`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:11.662844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:11.729854 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1418/2853] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:11.742210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.744034 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:11.801406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c2e60-64b5-428d-966e-5590eaebffac admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.805170 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f2d30-fcae-49a8-825c-9757d3b4385c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.805823 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f2d30-fcae-49a8-825c-9757d3b4385c service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.806104 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f2d30-fcae-49a8-825c-9757d3b4385c service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.806376 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f2d30-fcae-49a8-825c-9757d3b4385c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.815283 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f2d30-fcae-49a8-825c-9757d3b4385c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.816597 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f2d30-fcae-49a8-825c-9757d3b4385c service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.825714 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1436/2854] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:13.141470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.143468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:13.165160 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:13.207767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82753b9-fa0a-4fd5-9727-a8a8a0c33242 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.212371 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.213031 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.213313 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.213593 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] PATH_INFO: `/v3/users/c414197f142e4912b05abb8b0c6dc57b` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.214764 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c414197f142e4912b05abb8b0c6dc57b)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.217298 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.372691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c414197f142e4912b05abb8b0c6dc57b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:13.373293 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c414197f142e4912b05abb8b0c6dc57b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:13.461736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c414197f142e4912b05abb8b0c6dc57b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:13.474528 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c414197f142e4912b05abb8b0c6dc57b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:13.481449 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c414197f142e4912b05abb8b0c6dc57b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:13.488204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1419/2855] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:13 2022] DELETE /identity/v3/users/c414197f142e4912b05abb8b0c6dc57b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:13.493088 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.494555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.494555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.512764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:13.513998 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:13.571809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1437/2856] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:13.581567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.582999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:13.641437 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab9db-7dab-4520-b9d8-14050b159b62 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.645835 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d86771-7627-47ee-85af-536bc44df648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.646461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d86771-7627-47ee-85af-536bc44df648 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.646745 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d86771-7627-47ee-85af-536bc44df648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.647021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d86771-7627-47ee-85af-536bc44df648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.656199 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d86771-7627-47ee-85af-536bc44df648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.657740 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d86771-7627-47ee-85af-536bc44df648 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.667885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1420/2857] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:14.057693 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.059150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:14.071783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d997dd6-3798-4b3d-aaa4-c0985f1d5393 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.077795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.078478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.078895 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.079001 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] PATH_INFO: `/v3/projects/907705bd47c64492922c6cdde04c3407` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.080762 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] RBAC: Authorizing `identity:delete_project(project_id=907705bd47c64492922c6cdde04c3407)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.087657 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.183337 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '907705bd47c64492922c6cdde04c3407', 'request_id': 'req-189dc88d-4799-4137-ab88-e7b1b652b32c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.190344 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '907705bd47c64492922c6cdde04c3407', 'request_id': 'req-189dc88d-4799-4137-ab88-e7b1b652b32c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.191687 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:14.196568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '907705bd47c64492922c6cdde04c3407', 'request_id': 'req-189dc88d-4799-4137-ab88-e7b1b652b32c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.324066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] The token cache is being invalidate because project 907705bd47c64492922c6cdde04c3407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:14.324362 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.325017 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-189dc88d-4799-4137-ab88-e7b1b652b32c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.326525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1438/2858] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:14 2022] DELETE /identity/v3/projects/907705bd47c64492922c6cdde04c3407 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:14.332470 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76dc54-e8b5-4106-ba5d-330327ed4578 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.332951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76dc54-e8b5-4106-ba5d-330327ed4578 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.333169 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76dc54-e8b5-4106-ba5d-330327ed4578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.353527 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-ed76dc54-e8b5-4106-ba5d-330327ed4578 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:14.354738 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-ed76dc54-e8b5-4106-ba5d-330327ed4578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:14.397341 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:14.418089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1421/2859] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:14.426588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d8c72-c283-4b30-a71f-f8073fcd265d None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.426867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d8c72-c283-4b30-a71f-f8073fcd265d None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.427163 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d8c72-c283-4b30-a71f-f8073fcd265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.447210 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-557d8c72-c283-4b30-a71f-f8073fcd265d None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:14.448314 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-557d8c72-c283-4b30-a71f-f8073fcd265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:14.456151 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1439/2860] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:14.460707 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76dc54-e8b5-4106-ba5d-330327ed4578 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.470034 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76dc54-e8b5-4106-ba5d-330327ed4578 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.474699 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.475306 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.475587 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.475864 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.476599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:14.476906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.478496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.490879 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1422/2861] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:14.495047 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d8c72-c283-4b30-a71f-f8073fcd265d None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.504692 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d8c72-c283-4b30-a71f-f8073fcd265d None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.509265 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.509859 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.510135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.510402 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.511045 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.512969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.600405 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1440/2862] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:14 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:14.604507 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.615161 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035411-b2c5-4010-9ad5-dcdb2748ee6d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.669994 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.670636 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.670920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.671198 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.671968 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.674424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.770429 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1423/2863] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:14 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:14.774509 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.785433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e96aa-ee6d-446f-8bca-a49228f5a3ec admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.790127 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.790736 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.791018 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.791321 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.792024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.793662 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.804177 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1441/2864] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:14.807944 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.817997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d67044-04bf-421e-ae41-1055f962fdde admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.822634 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2955b8f-5121-40f4-b60e-f3cc7bee88d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.823233 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955b8f-5121-40f4-b60e-f3cc7bee88d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.823517 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955b8f-5121-40f4-b60e-f3cc7bee88d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.823793 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2955b8f-5121-40f4-b60e-f3cc7bee88d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.824516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2955b8f-5121-40f4-b60e-f3cc7bee88d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.826104 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2955b8f-5121-40f4-b60e-f3cc7bee88d5 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.836626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1424/2865] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:14.840446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.851072 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b1d6-29ac-4769-b46c-5f662962a684 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.855669 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.856311 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.856621 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.856899 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] PATH_INFO: `/v3/projects/49b6c4dd0c0d41759f6ed89af237a1e5/users/c2fa2467a8fe4809be05fdc8f378753c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.871829 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b6c4dd0c0d41759f6ed89af237a1e5, user_id=c2fa2467a8fe4809be05fdc8f378753c, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.874467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.976475 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1442/2866] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:14 2022] PUT /identity/v3/projects/49b6c4dd0c0d41759f6ed89af237a1e5/users/c2fa2467a8fe4809be05fdc8f378753c/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:14.988340 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e5599-28cb-4210-be1c-913b80ecb154 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.988662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e5599-28cb-4210-be1c-913b80ecb154 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.988934 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e5599-28cb-4210-be1c-913b80ecb154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.016892 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-f72e5599-28cb-4210-be1c-913b80ecb154 None None] MFA Rules not processed for user `c2fa2467a8fe4809be05fdc8f378753c`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:15.018829 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-f72e5599-28cb-4210-be1c-913b80ecb154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:15.081523 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1425/2867] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:15.085958 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.149986 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda36b-609b-4f1c-92e1-f3e51219fd6a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.154634 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.155246 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.155525 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.155798 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.156503 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.158407 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.280914 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1443/2868] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:15.285043 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e5599-28cb-4210-be1c-913b80ecb154 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.295691 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e5599-28cb-4210-be1c-913b80ecb154 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.329530 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:15.352236 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.358089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.358089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.358089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.358089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.358089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.416948 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1426/2869] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:15.420974 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.431585 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61713e2-5b60-4ee5-9520-b4e5ec03aaa4 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.436156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.436794 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.437075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.437342 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.438043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.440244 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.450913 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1444/2870] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:15.454812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.464980 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66648-01e0-4664-a4de-8306f15ef868 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.469565 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36087f17-aa3a-482b-b94d-c6afc9c44202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.470649 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36087f17-aa3a-482b-b94d-c6afc9c44202 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.470922 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36087f17-aa3a-482b-b94d-c6afc9c44202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.471185 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36087f17-aa3a-482b-b94d-c6afc9c44202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.471867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36087f17-aa3a-482b-b94d-c6afc9c44202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.473504 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36087f17-aa3a-482b-b94d-c6afc9c44202 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.483516 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1427/2871] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:15.488196 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.498197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9336d-c227-417c-bbbd-d389a9f28b65 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.503143 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.503792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.504116 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.504413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] PATH_INFO: `/v3/projects/36594aaf6b2345be8e6aa881506ac7ad/users/3a7f48674d3e4f51aac9ec4fc4195b72/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.518735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36594aaf6b2345be8e6aa881506ac7ad, user_id=3a7f48674d3e4f51aac9ec4fc4195b72, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.521367 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.564307 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1445/2872] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:15 2022] PUT /identity/v3/projects/36594aaf6b2345be8e6aa881506ac7ad/users/3a7f48674d3e4f51aac9ec4fc4195b72/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:15.580208 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602bf8-2154-4815-93d6-94dda8729493 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.580550 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602bf8-2154-4815-93d6-94dda8729493 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.580828 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f602bf8-2154-4815-93d6-94dda8729493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.609538 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-9f602bf8-2154-4815-93d6-94dda8729493 None None] MFA Rules not processed for user `3a7f48674d3e4f51aac9ec4fc4195b72`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:15.611354 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9f602bf8-2154-4815-93d6-94dda8729493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:15.673990 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1428/2873] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:15.678345 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.708519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:15.739949 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d84b0d-1103-4399-9767-59a7bcf84889 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.744568 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.749625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.749625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.749625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.749625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.749625 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.806766 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1446/2874] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:15.810574 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602bf8-2154-4815-93d6-94dda8729493 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.821344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f602bf8-2154-4815-93d6-94dda8729493 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.876957 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.877525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.877804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.878075 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.878739 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.880683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.965009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1429/2875] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:15.969194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.979969 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa081c-4ea6-42c5-95c4-ef14e079e703 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.984806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.988092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.988092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.988092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.988092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.988550 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.998060 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1447/2876] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:16.002202 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.012369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392552c0-dc2c-4bdc-b281-4bb77c4d4264 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.017622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.018307 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.018596 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.018893 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] PATH_INFO: `/v3/projects/a93eb240d545499d9aa63fa5333d6cbe/users/23ca90ddb5e7437293de0d4c57fd31cb/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.035176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93eb240d545499d9aa63fa5333d6cbe, user_id=23ca90ddb5e7437293de0d4c57fd31cb, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.037853 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.102614 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1430/2877] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:16 2022] PUT /identity/v3/projects/a93eb240d545499d9aa63fa5333d6cbe/users/23ca90ddb5e7437293de0d4c57fd31cb/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:16.106588 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.117181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c85d7a-21cf-4f30-a3bd-fdfaf46a800a admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.175357 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c850643-2b66-452a-8961-8cf99d5bf7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.176120 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c850643-2b66-452a-8961-8cf99d5bf7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.176433 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c850643-2b66-452a-8961-8cf99d5bf7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.176712 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c850643-2b66-452a-8961-8cf99d5bf7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.177406 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c850643-2b66-452a-8961-8cf99d5bf7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.179043 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c850643-2b66-452a-8961-8cf99d5bf7b0 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.189125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1448/2878] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:16.193149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.203515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4295701-1297-41ce-9771-7d7af5556f0c admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.208753 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.209419 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.209703 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.210096 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] PATH_INFO: `/v3/projects/a93eb240d545499d9aa63fa5333d6cbe/users/23ca90ddb5e7437293de0d4c57fd31cb/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.212077 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93eb240d545499d9aa63fa5333d6cbe, user_id=23ca90ddb5e7437293de0d4c57fd31cb, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.214746 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.221033 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:16.226899 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1431/2879] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:16 2022] PUT /identity/v3/projects/a93eb240d545499d9aa63fa5333d6cbe/users/23ca90ddb5e7437293de0d4c57fd31cb/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:16.236569 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190b95d-b9b7-4825-9cdc-df6bc418b336 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.236857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190b95d-b9b7-4825-9cdc-df6bc418b336 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.237134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190b95d-b9b7-4825-9cdc-df6bc418b336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.264536 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-8190b95d-b9b7-4825-9cdc-df6bc418b336 None None] MFA Rules not processed for user `23ca90ddb5e7437293de0d4c57fd31cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:16.265735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-8190b95d-b9b7-4825-9cdc-df6bc418b336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.337240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1449/2880] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:16.348489 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.350258 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.407339 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc517b-e3b5-4e1b-a642-0a3f8b834f1d admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.411406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.412024 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.412305 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.412670 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.469153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.470632 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.479421 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1432/2881] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:18.473211 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190b95d-b9b7-4825-9cdc-df6bc418b336 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.483197 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190b95d-b9b7-4825-9cdc-df6bc418b336 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.487504 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.488113 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.488400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.488691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.510333 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:18.544911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.552279 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.556086 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1450/2882] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:19.895796 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.904983 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c1fd1-ede9-432b-aa1c-74d050f81cf6 service cinder] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.908589 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fec7c-5f90-4948-9816-67d0afd3b457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.909197 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fec7c-5f90-4948-9816-67d0afd3b457 service cinder] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.909480 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fec7c-5f90-4948-9816-67d0afd3b457 service cinder] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.909758 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fec7c-5f90-4948-9816-67d0afd3b457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.919182 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fec7c-5f90-4948-9816-67d0afd3b457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.920631 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fec7c-5f90-4948-9816-67d0afd3b457 service cinder] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.930726 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1433/2883] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:20.853021 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:20.854589 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:20.920204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec185-7aeb-4659-83c1-598815382a5e service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:20.924874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:20.925512 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:20.925791 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:20.926070 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] PATH_INFO: `/v3/users/bf6bc49148e747108a1f1877ddbfae90` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:20.927236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6bc49148e747108a1f1877ddbfae90)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:20.929175 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:21.092519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6bc49148e747108a1f1877ddbfae90'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.093098 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6bc49148e747108a1f1877ddbfae90'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.147822 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6bc49148e747108a1f1877ddbfae90'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.161183 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6bc49148e747108a1f1877ddbfae90'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.161868 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6bc49148e747108a1f1877ddbfae90'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.175419 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1451/2884] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:20 2022] DELETE /identity/v3/users/bf6bc49148e747108a1f1877ddbfae90 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:21.179427 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:21.179906 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:21.180019 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:21.199604 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:21.200844 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:21.261787 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1434/2885] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:21.271520 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:21.272978 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:21.319650 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:21.332157 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66391cd-3f17-41cf-a9bd-b5a5e1647ad5 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:21.336081 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5d0ca-9f1a-4390-a4a0-e4f9af0e5266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:21.336746 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5d0ca-9f1a-4390-a4a0-e4f9af0e5266 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:21.337031 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5d0ca-9f1a-4390-a4a0-e4f9af0e5266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:21.337291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5d0ca-9f1a-4390-a4a0-e4f9af0e5266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:21.347820 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5d0ca-9f1a-4390-a4a0-e4f9af0e5266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:21.349373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5d0ca-9f1a-4390-a4a0-e4f9af0e5266 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:21.359468 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1452/2886] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:21.768225 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:21.769697 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:21.783350 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4fa23-eb6f-4c5c-ac0f-d567d556f87a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:21.789053 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:21.789953 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:21.790424 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:21.791195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] PATH_INFO: `/v3/projects/84c83082a73646afb8d4d33232e14201` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:21.792078 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c83082a73646afb8d4d33232e14201)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:21.796518 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:21.916097 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:21.925335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c83082a73646afb8d4d33232e14201', 'request_id': 'req-2b2244a1-ad7f-47fa-a776-fcb08275de9e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.931818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c83082a73646afb8d4d33232e14201', 'request_id': 'req-2b2244a1-ad7f-47fa-a776-fcb08275de9e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.937733 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c83082a73646afb8d4d33232e14201', 'request_id': 'req-2b2244a1-ad7f-47fa-a776-fcb08275de9e'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.998262 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] The token cache is being invalidate because project 84c83082a73646afb8d4d33232e14201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:21.998549 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.999162 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b2244a1-ad7f-47fa-a776-fcb08275de9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.000720 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1435/2887] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:21 2022] DELETE /identity/v3/projects/84c83082a73646afb8d4d33232e14201 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:22.006066 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147636-a42b-45c9-80fe-3b74d6229623 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.006561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147636-a42b-45c9-80fe-3b74d6229623 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.006682 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147636-a42b-45c9-80fe-3b74d6229623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.027153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-36147636-a42b-45c9-80fe-3b74d6229623 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:22.028424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-36147636-a42b-45c9-80fe-3b74d6229623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.087366 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1453/2888] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:22.096067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fde47c-c50f-4c86-8fe6-6913f589ca62 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.096463 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fde47c-c50f-4c86-8fe6-6913f589ca62 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.096812 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fde47c-c50f-4c86-8fe6-6913f589ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.116552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-17fde47c-c50f-4c86-8fe6-6913f589ca62 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:22.117769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-17fde47c-c50f-4c86-8fe6-6913f589ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.125875 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1436/2889] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:22.130200 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147636-a42b-45c9-80fe-3b74d6229623 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.139705 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147636-a42b-45c9-80fe-3b74d6229623 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.144871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.145479 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.145763 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.146036 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.146728 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:22.147030 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.148639 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.160696 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1454/2890] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:22.165181 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fde47c-c50f-4c86-8fe6-6913f589ca62 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.174673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fde47c-c50f-4c86-8fe6-6913f589ca62 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.179297 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.179960 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.180585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.180585 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.181196 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.183058 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.256795 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1437/2891] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:22.261014 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.271758 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda0b5f-60bd-4325-b239-dffb7e334537 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.326334 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.326942 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.327222 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.327499 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.328150 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.330775 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.439769 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1455/2892] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:22.443901 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.455320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3f3ef-819c-42fb-8bf3-48bc99b858a2 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.459891 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.460588 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.460903 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.461175 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.461863 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.463444 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.474880 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1438/2893] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:22.479648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.490075 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122d497-9d0f-4f79-b5ab-638ec4a88adb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.494809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e56ee-0a82-4f6a-872d-7f447957aedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.495418 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e56ee-0a82-4f6a-872d-7f447957aedf admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.495699 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e56ee-0a82-4f6a-872d-7f447957aedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.495976 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e56ee-0a82-4f6a-872d-7f447957aedf admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.496707 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e56ee-0a82-4f6a-872d-7f447957aedf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.498291 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e56ee-0a82-4f6a-872d-7f447957aedf admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.509413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1456/2894] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:22.513581 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.524210 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392e8a2-b122-45a8-b066-606071b2ba09 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.528969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.529693 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.529969 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.530242 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] PATH_INFO: `/v3/projects/155ced356cdb445c904f0eba6a929ebe/users/21324556c57e491db51f1a62f08ead79/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.544724 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=155ced356cdb445c904f0eba6a929ebe, user_id=21324556c57e491db51f1a62f08ead79, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.547406 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.616809 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1439/2895] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:22 2022] PUT /identity/v3/projects/155ced356cdb445c904f0eba6a929ebe/users/21324556c57e491db51f1a62f08ead79/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:22.621847 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.622135 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.622409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.641413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:22.644182 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.686165 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:22.701486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1457/2896] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:22.710214 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.711902 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.770113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b78845-a3ec-485a-b2e1-586cf69c8a8f admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.773789 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63182f50-aaa9-47d8-b95d-a19ada7ab058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.774413 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63182f50-aaa9-47d8-b95d-a19ada7ab058 service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.774692 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63182f50-aaa9-47d8-b95d-a19ada7ab058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.774965 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63182f50-aaa9-47d8-b95d-a19ada7ab058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.784536 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63182f50-aaa9-47d8-b95d-a19ada7ab058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.786717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63182f50-aaa9-47d8-b95d-a19ada7ab058 service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.796188 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1440/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:24.534074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:24.535524 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:24.548132 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3fe41-9616-45bd-aa21-301c92168bea None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:24.553655 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:24.554391 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:24.554748 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:24.555102 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] PATH_INFO: `/v3/users/c2fa2467a8fe4809be05fdc8f378753c` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:24.556409 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2fa2467a8fe4809be05fdc8f378753c)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:24.558400 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:24.882681 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2fa2467a8fe4809be05fdc8f378753c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:24.883346 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2fa2467a8fe4809be05fdc8f378753c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:24.954429 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2fa2467a8fe4809be05fdc8f378753c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:24.973708 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2fa2467a8fe4809be05fdc8f378753c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:24.975446 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2fa2467a8fe4809be05fdc8f378753c'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:24.991107 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1458/2898] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:24 2022] DELETE /identity/v3/users/c2fa2467a8fe4809be05fdc8f378753c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:24.995804 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:24.996360 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:24.996626 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:25.024335 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:25.026979 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:25.094638 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1441/2899] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:25.108519 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:25.110792 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:25.222545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af0a0-da81-45f3-86dc-9c85da31dcae admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:25.226517 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:25.227156 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:25.227706 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:25.227890 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:25.238863 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:25.240308 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:25.251194 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1459/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:25.741482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:25.743071 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:25.751889 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:25.760219 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09295445-a030-4c61-bae7-d10e0db4256a None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:25.765937 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:25.766683 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:25.767568 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:25.767904 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] PATH_INFO: `/v3/projects/49b6c4dd0c0d41759f6ed89af237a1e5` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:25.769286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b6c4dd0c0d41759f6ed89af237a1e5)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:25.771327 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:25.900576 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b6c4dd0c0d41759f6ed89af237a1e5', 'request_id': 'req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:25.912533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b6c4dd0c0d41759f6ed89af237a1e5', 'request_id': 'req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:25.916015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b6c4dd0c0d41759f6ed89af237a1e5', 'request_id': 'req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:25.993635 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] The token cache is being invalidate because project 49b6c4dd0c0d41759f6ed89af237a1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:25.993867 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:25.994461 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:25.995998 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1442/2901] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:25 2022] DELETE /identity/v3/projects/49b6c4dd0c0d41759f6ed89af237a1e5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:26.000282 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:26.012317 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96897a2e-3b20-4aeb-bed7-ab6d4a6392bf service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:26.090731 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:26.095813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:26.095813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:26.095813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] PATH_INFO: `/v3/users/3a7f48674d3e4f51aac9ec4fc4195b72` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:26.095813 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7f48674d3e4f51aac9ec4fc4195b72)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:26.097555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:26.104973 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:26.279989 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7f48674d3e4f51aac9ec4fc4195b72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:26.280742 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7f48674d3e4f51aac9ec4fc4195b72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:26.348236 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7f48674d3e4f51aac9ec4fc4195b72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:26.362195 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7f48674d3e4f51aac9ec4fc4195b72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:26.362867 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7f48674d3e4f51aac9ec4fc4195b72'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:26.381106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1460/2902] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:25 2022] DELETE /identity/v3/users/3a7f48674d3e4f51aac9ec4fc4195b72 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:26.831690 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:26.833329 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:26.852512 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70e740-5d3d-48d0-a9e6-9729ba55de7b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:26.923253 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:26.923910 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:26.924264 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:26.924575 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] PATH_INFO: `/v3/projects/36594aaf6b2345be8e6aa881506ac7ad` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:26.929108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36594aaf6b2345be8e6aa881506ac7ad)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:26.929108 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:27.053951 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36594aaf6b2345be8e6aa881506ac7ad', 'request_id': 'req-87899118-5288-49da-be5f-fc7c15dd0181'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.060810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36594aaf6b2345be8e6aa881506ac7ad', 'request_id': 'req-87899118-5288-49da-be5f-fc7c15dd0181'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.070067 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36594aaf6b2345be8e6aa881506ac7ad', 'request_id': 'req-87899118-5288-49da-be5f-fc7c15dd0181'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.148296 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] The token cache is being invalidate because project 36594aaf6b2345be8e6aa881506ac7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:27.152790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.152790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.152790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1443/2903] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:26 2022] DELETE /identity/v3/projects/36594aaf6b2345be8e6aa881506ac7ad => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:27.155456 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:27.156946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:27.171908 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906408a-b321-4ab8-931a-1bcf46d83142 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:27.240181 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:27.241172 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:27.241685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:27.242059 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] PATH_INFO: `/v3/users/23ca90ddb5e7437293de0d4c57fd31cb` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:27.243350 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ca90ddb5e7437293de0d4c57fd31cb)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:27.245457 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:27.396359 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ca90ddb5e7437293de0d4c57fd31cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.397051 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ca90ddb5e7437293de0d4c57fd31cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.462572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ca90ddb5e7437293de0d4c57fd31cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.484522 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ca90ddb5e7437293de0d4c57fd31cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.485260 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-7deb1df0-9f7c-46a4-8d05-615ab56359a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ca90ddb5e7437293de0d4c57fd31cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:27.498373 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1461/2904] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:27 2022] DELETE /identity/v3/users/23ca90ddb5e7437293de0d4c57fd31cb => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:28.047331 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.052091 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.062439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87899118-5288-49da-be5f-fc7c15dd0181 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.075978 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:28.133145 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.134344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.134907 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.135491 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] PATH_INFO: `/v3/projects/a93eb240d545499d9aa63fa5333d6cbe` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.137646 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93eb240d545499d9aa63fa5333d6cbe)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.140192 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.261895 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93eb240d545499d9aa63fa5333d6cbe', 'request_id': 'req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.271137 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93eb240d545499d9aa63fa5333d6cbe', 'request_id': 'req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.277779 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93eb240d545499d9aa63fa5333d6cbe', 'request_id': 'req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.383656 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] The token cache is being invalidate because project a93eb240d545499d9aa63fa5333d6cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:28.384525 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.386076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-49fa442a-ddcd-4d6e-a6d5-b3dab0940d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.388468 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1444/2905] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:28 2022] DELETE /identity/v3/projects/a93eb240d545499d9aa63fa5333d6cbe => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:28.393285 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070e478-2bf4-45c6-beee-15ae8c4144ed None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.393783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070e478-2bf4-45c6-beee-15ae8c4144ed None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.393889 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070e478-2bf4-45c6-beee-15ae8c4144ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.406442 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:28.428068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6070e478-2bf4-45c6-beee-15ae8c4144ed None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:28.428068 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6070e478-2bf4-45c6-beee-15ae8c4144ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.515046 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1462/2906] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:28.524509 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f79700-1a58-4cb3-a851-e72dea604fc4 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.530841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f79700-1a58-4cb3-a851-e72dea604fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.530841 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f79700-1a58-4cb3-a851-e72dea604fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.553735 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-60f79700-1a58-4cb3-a851-e72dea604fc4 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:28.554999 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-60f79700-1a58-4cb3-a851-e72dea604fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.565057 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1445/2907] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:28.569599 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070e478-2bf4-45c6-beee-15ae8c4144ed None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.588420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070e478-2bf4-45c6-beee-15ae8c4144ed None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.592897 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.593290 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.593662 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.601164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.601819 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.602106 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.602383 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] PATH_INFO: `/v3/domains` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.603101 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:28.603413 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.605082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.628622 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] MFA Rules not processed for user `21324556c57e491db51f1a62f08ead79`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:28.628902 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1463/2908] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:28.630332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.630502 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.640810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071337e4-be83-4be9-b5c4-8a0e796feefb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.646546 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.647365 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.647799 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.648067 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.649008 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.651020 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.704785 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1446/2909] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:28.715441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.716982 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.731613 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69d55a-71ed-4c1c-944d-cb9b0042c5bd None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.737440 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.738074 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.738357 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.738769 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.739417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1464/2910] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:28.739567 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.741485 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.743583 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.811136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57dd45-fbb6-4004-9aab-333b8a3ae7cb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.816511 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.817134 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.826980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.826980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] PATH_INFO: `/v3/users` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.826980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.826980 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.849711 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1447/2911] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:28.853790 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.863862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:28.904500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1465/2912] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:28.906966 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.916771 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a849831-b5d0-496c-a646-cf660c520bef admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.917398 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11f681-c02f-4644-8bdc-fb6f08554a55 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.921478 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.924626 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.924768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.925023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.925023 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.925239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.925239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.925239 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] PATH_INFO: `/v3/users` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.925541 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.925654 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.925772 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.926448 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.935785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1466/2913] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:28.940412 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.950486 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a4622-671a-40d1-9ec9-7faab245b416 admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.955871 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.959648 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.959956 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.960232 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] PATH_INFO: `/v3/roles` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.961141 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.964990 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.977117 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1467/2914] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:28.981186 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.992854 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c1778-dd43-4d27-bbae-5c161651a0da admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.997851 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b11c9-8c53-4004-b2f8-c6102ff9c030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.998575 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b11c9-8c53-4004-b2f8-c6102ff9c030 admin admin] REQUEST_METHOD: `PUT` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.998785 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b11c9-8c53-4004-b2f8-c6102ff9c030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.999058 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b11c9-8c53-4004-b2f8-c6102ff9c030 admin admin] PATH_INFO: `/v3/projects/f2aec663ad014da9ba1b2b144fdde597/users/fbfef3eacf5a4309b84aa3101e1ff189/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.008426 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1448/2915] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:29.010414 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.016464 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b11c9-8c53-4004-b2f8-c6102ff9c030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2aec663ad014da9ba1b2b144fdde597, user_id=fbfef3eacf5a4309b84aa3101e1ff189, role_id=3fa40e7c0efb4cdab3f6917f1a40e5e7)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.020089 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b11c9-8c53-4004-b2f8-c6102ff9c030 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.021768 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd5e4a-7bd6-49d6-8ba5-5e021863b747 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.026823 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.027584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.027947 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.028234 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.029021 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.031195 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.042560 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1449/2916] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:29.046820 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.057842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f94b4-6f2c-4298-87de-56bc396e0cd4 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.062997 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.064025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.064515 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.064799 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] PATH_INFO: `/v3/projects/466277dd6886445799294b508fff24ce/users/87453c33b50f4b0cbe2a090c3df066f5/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.081435 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=466277dd6886445799294b508fff24ce, user_id=87453c33b50f4b0cbe2a090c3df066f5, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.084079 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.106795 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1468/2917] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:28 2022] PUT /identity/v3/projects/f2aec663ad014da9ba1b2b144fdde597/users/fbfef3eacf5a4309b84aa3101e1ff189/roles/3fa40e7c0efb4cdab3f6917f1a40e5e7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:29.118668 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.119065 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.119533 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.144917 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:29.148620 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] MFA Rules not processed for user `fbfef3eacf5a4309b84aa3101e1ff189`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:29.149932 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:29.169076 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1450/2918] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:29 2022] PUT /identity/v3/projects/466277dd6886445799294b508fff24ce/users/87453c33b50f4b0cbe2a090c3df066f5/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:29.174330 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.225623 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1469/2919] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:29.244302 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fa379-0999-4151-8779-b1aed3e5a1cd admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.244467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.244467 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:29.246318 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.247232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.247521 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.247920 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] PATH_INFO: `/v3/roles` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.248776 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.251254 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.267153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1451/2920] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:29.268500 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.279784 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ffbed-2708-47b3-9a9c-23e011fc371b admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.285752 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007301a-be95-403d-9547-e80e2a92bab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.286441 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007301a-be95-403d-9547-e80e2a92bab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.286700 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007301a-be95-403d-9547-e80e2a92bab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.286985 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007301a-be95-403d-9547-e80e2a92bab7 admin admin] PATH_INFO: `/v3/projects/466277dd6886445799294b508fff24ce/users/87453c33b50f4b0cbe2a090c3df066f5/roles/7912a7addff34d9582eb2933d7110ebc` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.289709 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007301a-be95-403d-9547-e80e2a92bab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=466277dd6886445799294b508fff24ce, user_id=87453c33b50f4b0cbe2a090c3df066f5, role_id=7912a7addff34d9582eb2933d7110ebc)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.292118 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007301a-be95-403d-9547-e80e2a92bab7 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.305382 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1452/2921] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:29 2022] PUT /identity/v3/projects/466277dd6886445799294b508fff24ce/users/87453c33b50f4b0cbe2a090c3df066f5/roles/7912a7addff34d9582eb2933d7110ebc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:29.336033 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae4212-a4d2-4f6a-baa3-43814ac39b25 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.387190 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78776208cde94134b478ceee6cc70ec8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.387809 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.388092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.388368 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.448521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.450077 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.459911 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1470/2922] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:54:34.914584 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7db73-980c-4a43-a7c3-e5ecaf2f9d13 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:34.915431 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7db73-980c-4a43-a7c3-e5ecaf2f9d13 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:34.915722 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7db73-980c-4a43-a7c3-e5ecaf2f9d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:34.947133 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-6ce7db73-980c-4a43-a7c3-e5ecaf2f9d13 None None] MFA Rules not processed for user `87453c33b50f4b0cbe2a090c3df066f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:34.948542 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-6ce7db73-980c-4a43-a7c3-e5ecaf2f9d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:35.012788 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:35.098625 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1453/2923] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:35.109925 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:35.111500 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:35.129968 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:35.190526 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e6b3-000b-4f01-82b4-1cbb077b5f69 service cinder] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:35.195845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '813427e9e85a4b48a550a290211f39b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:35.196667 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:35.197054 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:35.197434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:35.285122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:35.286572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:35.298507 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1471/2924] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:35.570908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fad1a-01c5-421c-af97-99adb35fb395 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:35.570908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fad1a-01c5-421c-af97-99adb35fb395 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:35.570908 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fad1a-01c5-421c-af97-99adb35fb395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:35.608655 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-a16fad1a-01c5-421c-af97-99adb35fb395 None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:35.612840 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-a16fad1a-01c5-421c-af97-99adb35fb395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:35.690493 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1454/2925] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:54:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:35.705032 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:35.707532 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:35.791434 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73804514-e89b-4ae5-ab35-af4a68149c89 service glance] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:35.796595 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d41059-14e5-49d8-92dc-306e8854cf07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:35.797288 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41059-14e5-49d8-92dc-306e8854cf07 service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:35.797645 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41059-14e5-49d8-92dc-306e8854cf07 service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:35.797993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d41059-14e5-49d8-92dc-306e8854cf07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:35.807161 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41059-14e5-49d8-92dc-306e8854cf07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:35.808855 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d41059-14e5-49d8-92dc-306e8854cf07 service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:35.818783 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1472/2926] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:54:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:54:41.973149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fad1a-01c5-421c-af97-99adb35fb395 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:42.034286 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fad1a-01c5-421c-af97-99adb35fb395 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:42.038818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:42.039484 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:42.039763 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:42.040036 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] PATH_INFO: `/v3/users/fbfef3eacf5a4309b84aa3101e1ff189` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:42.041232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbfef3eacf5a4309b84aa3101e1ff189)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:42.043136 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:42.211232 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbfef3eacf5a4309b84aa3101e1ff189'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:42.211921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbfef3eacf5a4309b84aa3101e1ff189'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:42.268204 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbfef3eacf5a4309b84aa3101e1ff189'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:42.287098 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbfef3eacf5a4309b84aa3101e1ff189'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:42.292238 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:42.295436 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbfef3eacf5a4309b84aa3101e1ff189'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:42.305439 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1455/2927] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:41 2022] DELETE /identity/v3/users/fbfef3eacf5a4309b84aa3101e1ff189 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:42.315144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:42.315144 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:42.315803 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:42.343941 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] MFA Rules not processed for user `bf4d4f6d05b846a9b37142cd6195bb44`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:42.346024 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:42.408549 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=100 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:42.425204 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1473/2928] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:42.439073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:42.440842 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:42.507369 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cd138-0a58-4db3-a7a6-cc0ba2533b16 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:42.511164 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47e0ed-c95e-4a40-b3f1-23bb4416be3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:42.511848 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47e0ed-c95e-4a40-b3f1-23bb4416be3f service neutron] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:42.514403 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47e0ed-c95e-4a40-b3f1-23bb4416be3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:42.514681 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47e0ed-c95e-4a40-b3f1-23bb4416be3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:42.525007 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47e0ed-c95e-4a40-b3f1-23bb4416be3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:42.526405 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47e0ed-c95e-4a40-b3f1-23bb4416be3f service neutron] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:42.539511 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1456/2929] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:42.994587 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:43.001003 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:43.024873 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55008bf7-e7fe-4d1c-a30a-36b86b537644 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:43.034764 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:43.035815 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:43.036119 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:43.036420 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] PATH_INFO: `/v3/projects/f2aec663ad014da9ba1b2b144fdde597` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:43.037782 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2aec663ad014da9ba1b2b144fdde597)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:43.039697 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:43.138521 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2aec663ad014da9ba1b2b144fdde597', 'request_id': 'req-a99895c4-bed4-48e5-9e7e-a411574f472f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:43.146452 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2aec663ad014da9ba1b2b144fdde597', 'request_id': 'req-a99895c4-bed4-48e5-9e7e-a411574f472f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:43.152857 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2aec663ad014da9ba1b2b144fdde597', 'request_id': 'req-a99895c4-bed4-48e5-9e7e-a411574f472f'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:43.250577 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] The token cache is being invalidate because project f2aec663ad014da9ba1b2b144fdde597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:43.250845 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:43.251490 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:43.253289 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1474/2930] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:42 2022] DELETE /identity/v3/projects/f2aec663ad014da9ba1b2b144fdde597 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:45.764736 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:45.770935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:45.770935 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:45.786669 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] MFA Rules not processed for user `21324556c57e491db51f1a62f08ead79`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:45.787917 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:45.843391 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1457/2931] 10.222.0.47 () {58 vars in 958 bytes} [Wed Jul 27 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:46.093793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:46.095806 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:46.152214 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99895c4-bed4-48e5-9e7e-a411574f472f admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:46.155685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8b00b-9539-411e-8e7b-cd7f8e41d2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:46.156300 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b00b-9539-411e-8e7b-cd7f8e41d2ce service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:46.157136 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b00b-9539-411e-8e7b-cd7f8e41d2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:46.157417 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8b00b-9539-411e-8e7b-cd7f8e41d2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:46.159166 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-69f8b00b-9539-411e-8e7b-cd7f8e41d2ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:46.168816 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8b00b-9539-411e-8e7b-cd7f8e41d2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:46.170224 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8b00b-9539-411e-8e7b-cd7f8e41d2ce service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:46.178115 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1475/2932] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:47.649770 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:47.651352 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:47.724858 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5e038-17ef-441d-84d1-abcd1504ba56 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:47.729780 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:47.730390 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:47.730672 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:47.730942 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:47.733050 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:47.744344 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:47.745931 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:47.756496 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1458/2933] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:50.572164 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] REQUEST_METHOD: `POST` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:50.572665 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:50.572793 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:50.593153 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.auth.core [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:50.594903 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:50.653543 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1476/2934] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Jul 27 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:50.682068 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:50.683645 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:50.720244 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=100 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:50.760704 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0f5e0-4071-4097-8791-d3d9e1d64ff2 service placement] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:50.767422 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:50.768149 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:50.768578 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:50.768860 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:50.784551 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:50.788292 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:50.797975 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1459/2935] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:54:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:54:52.014964 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:52.017686 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:52.092685 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:52.158424 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be42f47-34ad-483d-91f9-f2773fe13b25 None None] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:52.165019 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:52.165720 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:52.165946 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:52.166252 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:52.168074 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:52.184414 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:52.186313 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:52.199711 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1477/2936] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:55:00.793885 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=18 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:55:02.290082 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=21 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:55:05.065513 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:55:05.077610 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4339e-9f72-423d-bdf9-d0e8ca0a7950 service swift] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:55:05.083810 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb68b48-b980-4dcf-a51c-a3c2bbb05ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '784880e276bf4d64ae785609a463403a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:55:05.084438 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb68b48-b980-4dcf-a51c-a3c2bbb05ec2 service nova] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:55:05.084718 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb68b48-b980-4dcf-a51c-a3c2bbb05ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:55:05.084989 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb68b48-b980-4dcf-a51c-a3c2bbb05ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:55:05.086894 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-4cb68b48-b980-4dcf-a51c-a3c2bbb05ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:55:05.099667 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb68b48-b980-4dcf-a51c-a3c2bbb05ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:55:05.103263 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb68b48-b980-4dcf-a51c-a3c2bbb05ec2 service nova] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:55:05.112126 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1460/2937] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:55:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:55:15.107599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=6 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:55:18.657294 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:55:18.666544 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5799-2c96-4c7e-ab7e-3d4c2497eac3 service nova] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:55:18.670335 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54aedfbb64c54c589003b335fde2a17b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:55:18.670937 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:55:18.671216 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:55:18.671488 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:55:18.673856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:55:18.740092 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:55:18.741555 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:55:18.751122 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1478/2938] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:55:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:55:28.746750 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=30 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:56:49.215552 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] REQUEST_METHOD: `POST` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:56:49.216025 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:56:49.216113 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:56:49.236862 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.auth.core [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] MFA Rules not processed for user `f247bde98344474ea5a62b4fa04464b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:56:49.238073 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:56:49.245967 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1461/2939] 10.222.0.47 () {58 vars in 967 bytes} [Wed Jul 27 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:56:49.285527 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:56:49.297835 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c80ad-71e9-4b77-9997-8af285e0c451 service neutron] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:56:49.302874 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcebfbf0b3084929bc33887467695b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:56:49.303496 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] REQUEST_METHOD: `GET` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:56:49.304404 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:56:49.304690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:56:49.313856 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:56:49.315561 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:56:49.324061 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1479/2940] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Jul 27 20:56:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:56:57.353300 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:56:57.355015 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:56:57.420602 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca3daf-7b8e-4c40-821c-ed275f4d5d85 None None] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:56:57.425237 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:56:57.425874 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:56:57.426154 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:56:57.426432 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] PATH_INFO: `/v3/users/21324556c57e491db51f1a62f08ead79` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:56:57.427616 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21324556c57e491db51f1a62f08ead79)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:56:57.429599 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:56:57.624446 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21324556c57e491db51f1a62f08ead79'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:57.625009 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21324556c57e491db51f1a62f08ead79'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:57.701767 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21324556c57e491db51f1a62f08ead79'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:57.713815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21324556c57e491db51f1a62f08ead79'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:57.720482 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21324556c57e491db51f1a62f08ead79'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:57.726433 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1462/2941] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:56:57 2022] DELETE /identity/v3/users/21324556c57e491db51f1a62f08ead79 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:56:57.971735 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:56:57.983649 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a443a-8977-43a0-8a6c-ab20253256ad service swift] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:56:58.042690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:56:58.043332 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:56:58.043612 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:56:58.043886 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] PATH_INFO: `/v3/projects/155ced356cdb445c904f0eba6a929ebe` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:56:58.045084 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=155ced356cdb445c904f0eba6a929ebe)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:56:58.046993 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:56:58.187302 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155ced356cdb445c904f0eba6a929ebe', 'request_id': 'req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.193690 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155ced356cdb445c904f0eba6a929ebe', 'request_id': 'req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.199125 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155ced356cdb445c904f0eba6a929ebe', 'request_id': 'req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.266572 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] The token cache is being invalidate because project 155ced356cdb445c904f0eba6a929ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:56:58.266862 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.267485 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.269041 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1480/2942] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:56:57 2022] DELETE /identity/v3/projects/155ced356cdb445c904f0eba6a929ebe => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:56:58.273430 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:56:58.274921 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:56:58.287909 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38670d76-86bb-43c1-8a87-b6d8c6b94241 admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:56:58.344460 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:56:58.345107 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:56:58.345392 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:56:58.345673 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] PATH_INFO: `/v3/users/87453c33b50f4b0cbe2a090c3df066f5` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:56:58.346857 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=87453c33b50f4b0cbe2a090c3df066f5)` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:56:58.348818 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:56:58.363172 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:56:58.544737 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87453c33b50f4b0cbe2a090c3df066f5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.545343 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87453c33b50f4b0cbe2a090c3df066f5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.730815 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87453c33b50f4b0cbe2a090c3df066f5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.743811 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87453c33b50f4b0cbe2a090c3df066f5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.750629 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.notifications [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87453c33b50f4b0cbe2a090c3df066f5'} {{(pid=80335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:58.756533 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1463/2943] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Jul 27 20:56:58 2022] DELETE /identity/v3/users/87453c33b50f4b0cbe2a090c3df066f5 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:56:59.006052 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Authenticating user token {{(pid=80336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:56:59.008148 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.fernet_utils [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:56:59.021716 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1751b8f-5071-46f7-a1ad-4fc55e4c29cb admin admin] Validating token access rules against request {{(pid=80336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:56:59.081121 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf4d4f6d05b846a9b37142cd6195bb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '285f06e24b5441ccbaca698e1fbd1162', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:56:59.081679 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:56:59.081979 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:56:59.082234 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] PATH_INFO: `/v3/projects/466277dd6886445799294b508fff24ce` {{(pid=80336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:56:59.083380 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466277dd6886445799294b508fff24ce)` {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:56:59.085355 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] RBAC: Authorization granted {{(pid=80336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:56:59.105833 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:56:59.182691 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466277dd6886445799294b508fff24ce', 'request_id': 'req-556022cb-a3af-4fb9-83c8-b2f6e97fa119'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:59.188810 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466277dd6886445799294b508fff24ce', 'request_id': 'req-556022cb-a3af-4fb9-83c8-b2f6e97fa119'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:59.194817 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466277dd6886445799294b508fff24ce', 'request_id': 'req-556022cb-a3af-4fb9-83c8-b2f6e97fa119'} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:59.272226 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] The token cache is being invalidate because project 466277dd6886445799294b508fff24ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:56:59.272545 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:59.273171 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG keystone.notifications [None req-556022cb-a3af-4fb9-83c8-b2f6e97fa119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:56:59.274680 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: [pid: 80336|app: 0|req: 1481/2944] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Jul 27 20:56:59 2022] DELETE /identity/v3/projects/466277dd6886445799294b508fff24ce => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:57:08.753048 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:57:09.269240 dv-cinder-851249-1-iscsi devstack@keystone.service[80336]: DEBUG dbcounter [-] [80336] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:58:05.584676 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Authenticating user token {{(pid=80335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:58:05.586089 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:58:05.655712 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dce60-d9eb-4bcd-b88a-b5d120d5f2af admin admin] Validating token access rules against request {{(pid=80335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:58:05.714228 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58451f5c-4ce8-444a-aaa7-630bcb616fb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328ec1aac43349b28a72d5df19fd2cdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '177080d4a05f46f1b40d6f62a83f7b79', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:58:05.714839 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58451f5c-4ce8-444a-aaa7-630bcb616fb2 service placement] REQUEST_METHOD: `GET` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:58:05.715121 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58451f5c-4ce8-444a-aaa7-630bcb616fb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:58:05.715397 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58451f5c-4ce8-444a-aaa7-630bcb616fb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:58:05.717176 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.fernet_utils [None req-58451f5c-4ce8-444a-aaa7-630bcb616fb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:58:05.729153 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58451f5c-4ce8-444a-aaa7-630bcb616fb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:58:05.730717 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58451f5c-4ce8-444a-aaa7-630bcb616fb2 service placement] RBAC: Authorization granted {{(pid=80335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:58:05.740658 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: [pid: 80335|app: 0|req: 1464/2945] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Jul 27 20:58:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:58:15.736320 dv-cinder-851249-1-iscsi devstack@keystone.service[80335]: DEBUG dbcounter [-] [80335] Writing DB stats keystone:SELECT=58 {{(pid=80335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}